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CHAPTER  1 
INTRODUCTION 

1.1     Problem  Statement 

In  a  1960  survey  of  nine  Air  Force  missile  systems,  it  was  found  that 
human  error  contributed  from  20  to  53  percent  to  system  unreliability  (26). 
Another  study  (24)  investigated  a  large  number  of  production  defects  at  the 
plant  of  a  prime  contractor  for  the  Atomic  Energy  Commission  and  it  was 
determined  that  82  percent  of  defects  found  by  inspectors  could  be  directly 
attributed  to  human  error.     The  above  figures  demonstrate  that  human  per- 
formance has  a  significant  impact  on  the  reliability  of  a  system.     Because 
of  this,  a  concentrated  effort  has  been  made  in  the  past  25  years  to  combine 
human  reliability  values  with  hardware  reliability  figures  to  obtain  an 
overall   reliability  index  for  the  man-machine  system. 

Numerous  human  reliability  models  have  been  formulated  that  attempt 
to  assign  reliability  values  to  an  operator's  performance  in  a  man-machine 
system.     Some  of  these  models  attempt  to  demonstrate  that  relationships 
exist  between  the  level  of  operator  performance  and  factors  such  as:   amount 
of  supervision,  working  environment,  willingness  to  work,  etc.     Very  seldom, 
and  in  most  cases  never,  do  any  of  these  human  reliability  models  address 
the  impact  of  an  operator's  rate  of  learning  on  the  human  performance  level 
and  subsequently  on  the  reliability  of  the  system. 

The  problem  that  will  be  addressed  by  this  study  is  the  impact  that 
changing  rates  of  learning  have  on  an  operator's  performance  level.     This 
change  in  an  operator's  performance  for  various  time  increments  will  be 


found  by  utilizing  a  Learn-Forget-Learn  (LFL)  curve  and  two  learning  curves. 
The  learning  curves  and  LFL  curve  will  be  developed  using  data  from  a  previous 
study  of  learning.  A  simulation  model,  developed  by  another  researcher  (16), 
which  quantifies  System  Effectiveness  (or  reliability)  will  be  used  in  con- 
junction with  the  results  from  the  two  curve  types  (learning  &  LFL)  to  study 
the  impact  that  learning  has  on  the  system  reliability  index.  The  sensi- 
tivity of  the  system  reliability  index  to  changing  rates  of  learning  will  be 
calculated,  analyzed  and  discussed.  It  will  be  shown  that  operator  learning 
does  have  an  effect  on  the  total  reliability  value  of  the  system. 

1 .2  Purpose 

The  main  purpose  of  this  study  is  to  demonstrate  that  operator  learning 
for  various  increments  of  time  will  have  a  significant  impact  on  operator 
performance  and  also,  therefore,  on  System  Effectiveness.  The  secondary 
purpose  is  to  outline  the  requirement  for  additional  and  more  thorough 
research  in  the  area  of  operator  learning  and  its  subsequent  impact  on  the 
reliability  of  the  overall  man-machine  system. 

The  study  is  organized  so  as  to  lend  support  to  the  main  theme  that 
was  outlined  above.  The  literature  survey,  which  follows  this  section, 
reviews  a  number  of  human  reliability  models  that  have  been  proposed. 
Only  two  of  these  models  address  the  aspect  of  operator  learning/training 
to  any  great  detail.  The  remaining  models  make  no  reference  to  operator 
training/learning  and  their  impact  on  the  level  of  human  performance.  An 
in  depth  comparison  of  various  models  will  be  presented  in  a  table  that 
will  outline  the  positive  and  negative  aspects  of  the  models.  The  liter- 
ature survey  also  contains  a  review  of  articles  that  pertain  to  learning 


curves.  The  uses  and  methodology  of  learning  curves  will  be  presented 
along  with  examples  of  typical  curves.  The  graphical  and  mathematical 
representation  of  the  learning  curves  will  also  be  discussed.  The  liter- 
ature survey  also  will  outline  the  methodology  of  LFL  curves,  but  only  one 
reference  was  discovered  which  attempted  to  explain  and  discuss  this  type 
of  curve.  The  techniques  and  conclusions  of  this  reference  will  be  out- 
lined in  the  literature  survey. 

Chapter  2  of  this  report  is  devoted  to  the  development  of  two  learning 
curves  and  an  LFL  curve.  Data  obtained  from  one  of  the  references  on 
learning  curves  will  be  used  as  a  basis  to  plot  two  different  learning 
curves.  The  LFL  curve  will  be  plotted  using  the  data  presented  in  the 
lone  LFL  curve  reference.  The  characteristics  of  each  curve  will  be  ex- 
plained and  mathematical  expressions  for  the  three  curves  will  be  developed. 

Chapter  3  presents  a  summary  and  explanation  of  a  System  Effectiveness 
model  which  has  been  developed  by  another  researcher  (16).  The  "personnel" 
term  associated  with  the  equation  for  this  model  will  be  analyzed  in  more 
detail,  especially  in  relation  to  learning.  The  effects  on  this  "personnel" 
or,  preferably,  "operator"  term  for  changing  rates  of  learning,  obtained 
from  the  respective  curves,  will  be  analyzed  in  terms  of  operator  performance. 
The  sensitivity  of  the  System  Effectiveness  model  to  these  fluctuating  levels 
of  operator  performance  then  will  be  studied  and  discussed.  Conclusions  then 
will  be  formulated  concerning  the  impact  of  operator  learning  on  the  System 
Effectiveness  model. 

The  last  section  of  the  study  will  outline  the  requirement  for  additional 
research  in  the  area  of  operator  learning  and  its  effect  on  the  reliability 


of  a  man-machine  system.     The  question  of  what  impact  does  operator 
learning  have  on  operator  performance  is  of  critical   importance  to  system 
reliability. 

1 .3    Literature  Survey 

This  study's  three  main  areas  of  interest  (human  reliability  models, 
learning  curves  and  LFL  curves)  will  be  addressed  separately  in  this 
survey  of  the  literature.     There  are  abundant  references  in  the  literature 
which  pertain  to  the  prediction  of  human  performance  in  man-machine  systems. 
A  summary  of  a  representative  and  well   known  method  for  predicting  human 
performance,  THERP,  will  be  presented.     There  are  also  numerous  articles 
devoted  to  learning  curves,  but  references  related  to  Learn-Forget-Learn 
(LFL)  curves  are  very  few  in  number,  almost  to  the  point  of  being  non- 
existant. 

1.3.1     Human  Reliability  Models 
1.3.1.1     Early  Studies 

In  the  past,  reliability  figures  were  calculated  for  a  man-machine 
system  based  solely  on  the  machine  component  of  the  system.     The  human 
component  was  assumed  to  be  totally  reliable  and  no  provisions  were  in- 
cluded in  the  models  to  account  for  human  unreliability.     Later  on  it  was 
determined  that  the  human  aspects  of  man-machine  systems  contributed  greatly 
to  the  system  unreliability,  in  some  instances  even  more  so  than  the  equip- 
ment component  £27].     After  this  discovery,  much  more  emphasis  was  placed 
on  predicting  human  performance  in  a  system. 


One  of  the  earliest  man-machine  reliability  studies  in  which  human 
error  rates  were  estimated  and  related  to  estimates  of  equipment  malfunction 
rates  was  done  in  1952  by  an  electronics  engineer  and  a  mathematician  at 
Sandia  Corporation  (27).  The  treatment  of  human  error  in  this  1952  study  of 
an  aircraft  nuclear  weapon  system  was  crude.  Only  those  errors  which 
would  directly  reduce  system  reliability  without  any  other  equipment  failure 
or  human  error  being  involved  were  studied.  The  estimates  of  human  error 
were  included  in  the  overall  system  reliability  equation  and  were  treated  in 
the  same  manner  as  estimates  of  failures  rates  for  other  system  factors. 

Later  studies  became  more  refined  in  regards  to  the  quantitative 
methods  utilized  for  evaluating  human  performance  and  its  relationship  to 
man-machine  system  performance.  One  report  recommended  a)  making  rough 
estimates  of  the  probability  of  successful  completion  of  each  sub-task  in 
a  system  and  then  b)  combining  the  probabilities  to  obtain  the  overall 
-reliability  of  the  system  (32).  Another  researcher  pointed  out  that  it  was 
necessary  to  treat  those  rough  error  rate  estimates,  mentioned  in  the  above 
study,  differentially  according  to  their  importance  to  system  performance  (23). 
He  defined  task  criticality  in  quantitative  terms  related  to  the  effect  of 
unsuccessful  task  completion  upon  system  success.  Eventually,  more 
sophisticated  models  were  developed  to  predict  human  performance  reliability 
more  accurately.  One  of  these  methods  was  called  Technique  for  Human  Error 
Rate  Prediction  (THERP). 


1.3.1.2  Technique  for  Human  Error  Rate  Prediction  (THERP). 

THERP  is  one  of  the  best  known  methods  developed  to  quantify  human 
performance.  In  1961,  Swain  (28)  developed  this  method  for  evaluating  the 
human  error  contribution  to  system  degradation.  The  following  discussion 
of  THERP  is  almost  entirely  from  a  paper  entitled  "Methods  of  Predicting 
Human  Reliability  in  Man-Machine  Systems"  by  David  Meister  of  the  Bunker- 
Ramo  Corporation  (17). 

THERP  has  been  used  primarily  to  provide  quantitative  predictions  of 
system  degradation  resulting  from  human  errors  in  association  with  equipment 
reliability,  operational  procedures,  and  other  system  characteristics  which 
influence  human  behavior.  THERP  is  an  iterative  procedure  that  consists  of 
five  steps  which  are  repeated,  not  always  in  the  same  order,  until  system 
degradation  resulting  from  human  error  is  at  an  acceptable  level.  The 
five  steps  are  listed  below. 

"(1)  Define  the  system  or  subsystem  failure  which  is  to  be 
evaluated. 

(2)  Identify  and  list  all  the  human  operations  performed  and  their 
relationships  to  system  tasks  and  functions. 

(3)  Predict  error  rates  for  each  human  operation  or  group  of 
operations  pertinent  to  the  evaluation. 

(4)  Determine  the  effect  on  human  errors  on  the  system. 

(5)  Recommend  changes  as  necessary  to  reduce  the  system  or  subsystem 
failure  rate  as  a  consequence  of  the  estimated  effects  on  the 
recommended  changes." 


Swain  (29)  points  out  that  "the  steps  are  typical  of  the  usual  system 
reliability  study  if  one  substitutes  'hardware'  for  'human'." 

The  goals  of  this  technique  are  listed  by  Meister  in  another  report 
entitled  "Comparative  Analysis  of  Human  Reliability  Models"  (18).  They 

are: 

"(1)  To  derive  'quantitative  estimates  of  the  degradation  to  a 
man-machine  system  resulting  from  human  error.' 

(2)  Or,  'to  evaluate  the  human  error  contribution  to  systems 
degradation. ' 

(3)  To  predict  human  error  rates. 

(4)  To  determine  those  design  changes  to  the  system  necessitated 
by  the  system  failure  rate." 

One  of  the  assumptions  associated  with  THERP,  as  listed  in  (18)  by 
Meister,  is: 

"THERP  takes  into  account  various  psychological  and  physiological 
stresses,  training,  motivation  and  situational  factors.  These 
are  called  Performance  Shaping  Factors  (PSF)  and  they  are  very 
subjective  in  their  application." 

In  regard  to  the  above  assumption,  Meister  makes  the  following 
comment: 

"These  factors,  i.e.  PSF,  must  be  taken  into  account  in  the  gathering 
of  error  rate  data  and  the  error  estimates  derived  should  be  modified 
in  accordance  with  the  presumed  effect  of  these  factors  on  performance. 
One  difficulty  that  arises,  however,  in  accounting  for  these  molar 
factors  on  performance  is  the  difficulty  of  recognizing  their  influence 
and  estimating  the  extent  of  that  influence." 

This  statement  by  Meister  embodies  the  purpose  of  this  study,  i.e.  to 

recognize  the  influence  and  estimate  the  extent  of  that  influence  on  human 


performance  caused  by  the  so-called  Performance  Shaping  Factors,  specifically, 
the  factor  of  learning.     THERP  has  no  specific  provisions  to  handle  varying 
levels  of  operator  learning  nor  to  predict  the  impact  that  these  learning 
levels  have  on  human  performance.     This  study  will  hopefully  demonstrate  the 
effect  that  learning  has  on  operator  performance. 

1.3.1.3    Comparison  of  Human  Reliability  Models 

Numerous  human  reliability  models  have  been  formulated  that  attempt 
to  predict  operator  performance  levels  in  man-machine  systems.     Davis  Meister, 
in  the  report,  "Comparative  Analysis  of  Human  Reliability  Models,"  summarized 
and  characterized  18  human  reliability  models  (18).     Table  1.1  is  an 
abbreviated  version  of  Meister's   "Summary  of  Model   Characteristics"  which 
can  be  found  on  page  414  of  (18).     It  should  be  noted  that  for  the  table's 
sub-category  of  "Selection/Training"  only  two  models,  the  Human  Operator 
Simulator  (HOS)  and  the  Personnel   Reliability  Index,  meet  the  criteria 
established  for  that  sub-category  by  Meister.     For  a  complete  description 
of  all  criteria  used  in  the  table,  consult  pages  413  through  425  of  (18). 

Meister  makes  the  following  remarks  concerning  the  sub-category  of 

"Sel ecti  on/Trai  ni  ng" : 

"Most  of  the  methods  possess  little  or  no  capability  in  the  areas  of 
manpower  selection  and  training  despite  the  fact  that  claims  for 
these  capabilities  are  often  made.     We  feel  that  to  be  sensitive  to 
training,  a  model  must  indicate  what  capabilities  should  be  trained, 
rather  than  merely  that  additional  training  is  required.     On  that 
basis  only  a  few  of  the  models,  i.e.,  the  personnel   reliability 
technique  of  Siegel  and  Wherry's  HOS,  seem  to  possess  this  sensitivity. 
It  may  be  that  the  majority  of  the  models  available  do  not  include 
parameters  which  are  sensitive  to  the  factor  of  training  or  it  may 
be  that  a  distinctly  different  type  of  model   is  required. 

This  comparison  of  human  reliability  models  points  out  that  very  little 

work  has  been  done  in  the  area  of  training  and  learning  with  their  attendant 
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impact  on  human  performance.  This  study  will  demonstrate  that  operator 
learning  has  a  significant  effect  on  human  performance. 

1.3.2  Learning  Curves 

In  1936  Wright  (33)  published  the  first  article  that  formulated  the 

theory  of  learning  curves.  He  noted  a  continuous  improvement  in  labor 

cost  in  the  manufacture  of  airplanes  as  the  workers  repeated  their  tasks. 

From  his  observations  and  study  in  the  aircraft  industry,  he  developed 

the  basic  learning  curve  theorem  which  can  be  stated  as: 

"For  any  operation  which  is  repeated,  the  time  of  the  operation 
will  decrease  by  a  fixed  fraction,  known  as  the  reduction  fraction, 
each  time  the  number  of  operations  doubles." 

Learning  curves  are  applicable  to  many  aspects  of  production  planning 
and  control.  They  can  be  used  to  predict  the  cost  per  unit  of  production, 
offer  quantity  discounts,  and  establish  selling  price.  Learning  curves 
also  influence  delivery  schedules,  set  labor  standards,  and  measure  shop 
efficiency  (2).  They  can  also  be  utilized  for  establishing  costs  of  manu- 
facture and  determining  labor  requirements. 

The  learning  curve  is  actually  a  line  on  a  graph  which  demonstrates 
the  reduction  of  time  in  any  repetitive  operation.  Two  facts  concerning  the 
use  of  learning  curves  are  important:  (1)  The  time  required  to  do  a  job 
will  decrease  each  time  the  job  is  repeated.  (2)  The  amount  of  decrease 
will  be  less  with  each  successive  unit. 

The  curve  may  be  presented  on  any  type  graph  paper  but  is  more  commonly 
portrayed  on  log-log  graph  paper.  When  plotted  on  arithmetic  graph  paper, 
the  shape  of  a  typical  learning  will  be  exponential  as  demonstrated  in 
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Figure  1.1.     It  can  be  noted  that  the  curve  possesses  the  characteristics 
of  an  initial   rapid  fall   followed  by  a  flattening  of  the  curve  and  after  a 
relative  small  number  of  repetitions,  the  rate  of  improvement  is  small. 
If  log-log  graph  paper  is  used  instead  of  arithmetic  graph  paper,  a 
straight  line  is  presented.     Figure  1.2  is  identical  with  Figure  1.1  except 
that  the  points  have  been  plotted  on  log-log  paper.     The  nature  of  log-log 
scales  permits  the  inclusion  of  many  repetitions  or  long  periods  of  time 
which  would  be  impossible  with  arithmetic  graph  paper.     On  log-log  paper, 
the  distance  between  doubled  quantities  is  equal.     This  fact  coupled  with 
the  learning  curve  theorem  is  why  the  plot  of  a  learning  curve  on  log-log 
paper  is  linear. 

The  learning  curve  is  a  power  curve  of  the  form: 


+    -  +  «"m 


where: 


t  =  the  time  of  operation  number  n 
t-.  =  the  time  of  the  first  operation 

n  =  number  of  repetitions 

m  =  slope  of  the  curve 

When  reduced  to  logarithmic  form,  this  equation  is  represented  by  the 
linear  equation:  log  t  =  log  t-,  -  m  log  n.  The  slope  of  the  line,  m, 
is  frequently  called  the  reduction  fraction  and  it  represents  the  rate 
of  learning.  The  reduction  fraction  usually  varies  between  .7  and  .95 
depending  on  the  proportion  of  labor  in  the  task  which  is  man-controlled  (8) 
The  complexity  of  the  task  and  human  motivation  are  also  factors  which 
affect  the  reduction  fraction. 


12 


4    6    8    10   12   14   16   18   20 

Number  of  Repetitions 
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EXPONENTIAL  LEARNING  CURVE  PLOTTED 
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FIGURE  1,2 
EXPONENTIAL  LEARNING  CURVE  PLOTTED 
ON  LOG-LOG  COORDINATES 


14 


Numerous  other  equations,  more  complex  and  involved  than  the  above 
equation  have  been  developed  to  express  the  learning  curve  theory  (31), 
(5)  and  (15).     One  of  the  equations  fits  an  S-curve  to  learning  phenomenon 
while  another  gives  an  expression  for  a  more  complex  exponential  curve. 

Carlson  and  Rowe  (6)  advocate  that  a  learning  curve,  plotted  on 
arithmetic  graph  paper,  will  have  an  S-shape  instead  of  the  exponential 
shape  proposed  by  Wright  (33).     See  Figure  1.3  for  an  example  of  the 
S-shaped  curve.     They  maintain  that  the  "incipient"  phase  generally  in- 
volves little  improvement  because  the  worker  is  getting  accustomed  to  the 
shop  setup,  tooling,  instructions,  workplace  arrangement,  and  the  conditions 
of  the  process.     The  second  phase,   "learning,"  is  where  most  of  the  im- 
provement takes  place  because  this  phase  includes  the  reduction  in  errors, 
development  of  a  work  pattern,  and  rearrangement  of  the  workplace.     The 
third  phase,   "maturity,"  represents  a  limit  to  improvement  because  some 
learning  still  takes  place  but  at  a  much  slower  rate  and  becomes  asymptotic  to 
the  1 imit. 

Numerous  discussions  have  taken  place  concerning  the  advantages  and 
disadvantages  of  Wright's  simple  equation  compared  to  the  more  complex 
expressions.     It  has  been  pointed  out  that  deficiencies  exist  in  the 
practical  use  of  the  power  form  model  of  Wright  (5).     Two  of  these  defici- 
encies are  the  model's  ultimate  asymptote  of  zero  and  the  infinite  learning 
period  i.e.   learning  rate  is  assumed  to  be  constant.     The  advantages  of 
Wright's  equation  are  its  simplicity  and  ease  of  calculations.     Also,  it  is 
more  easily  understood  by  management  than  the  more  complex  models.     Even 
though  the  disadvantages  of  Wright's  equation  are  significant,  the  consensus 


15 


100% 


CD 
U 

c 

03 


W 

D. 


50%  • 


Incipient/ 

/  Learning 

""^Maturity 

Number  of  Units 


FIGURE  1.3 
S-SHAPED  LEARNING  CURVE 
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has  been  that  the  simple  straight  line  on  log-log  paper  is  best  (11). 
Corlett  and  Morcombe  (8)  state  that  there  are  not  many  industrial 
studies  reported  of  the  use  of  learning  curves  in  the  field  of  training, 
but  it  is  in  training,  where  learning  is  taking  place  continually,  that  it 
should  have  the  most  applications.  This  study  will  apply  two  different 
learning  curve  equations  to  ascertain  the  performance  level  of  an  operator 
over  various  periods  of  time.  These  various  performance  levels  will  then 
be  utilized  in  an  equation  for  System  Effectiveness,  and  the  sensitivity 
of  the  System  Effectiveness  index  to  the  changing  performance  levels  will 
be  studied. 

1.3.3  LFL  Curves 

Very  little  research  has  been  done  in  the  area  of  Learn-Forget-Learn 
(LFL)  curves  and  their  impact  on  human  performance.  LFL  curves  usually 
have  a  saw-tooth  shape  as  can  be  seen  in  Figure  1.4.  This  shape  is  the 
result  of  an  operator  learning  a  particular  task  for  a  certain  time  period 
and  then  having  that  learning  interrupted  by  some  event  which  takes  him 
away  from  the  task.  In  all  probability,  he  will  forget  a  portion  of  what 
he  had  originally  learned,  and  his  performance  on  the  original  task  will 
decline.  This  sequence  of  events  account  for  the  curve  shape,  i.e.,  the 
initial  learning  is  depicted  as  a  gradual  increase  in  the  curve  followed 
by  a  more  pronounced  increase,  but  when  the  interruption  of  learning  occurs, 
the  curve  drops  off  and  operator  performance  decreases.  When  the  operator 
returns  to  the  task  after  the  interruption,  his  performance  starts  to  in- 
crease again  as  the  curve  begins  to  climb.  The  learning/ forgetting  curve 
explained  above  has  been  proposed  by  Carlson  and  Rowe  (6). 
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It  should  be  noted  that  the  forgetting  portion  of  Figure  1.4  shows  a 
rapid  initial  decrease  in  performance  followed  by  a  gradual  leveling  off 
as  a  function  of  the  interruption  interval  period.  Also,  the  rate  and 
amount  of  forgetting  decreases  as  an  increased  number  of  units  are  com- 
pleted before  an  interruption  occurs.  These  two  attributes  of  forgetting 
curves  demonstrate  that  the  amount  of  forgetting  and  the  corresponding  level 
of  performance  are  a  function  of  both  the  performance  at  the  time  the  task 
was  interrupted  and  the  length  of  the  interruption. 

From  the  above  discussion,  it  can  be  deduced  that  an  LFL  is  a  combin- 
ation of  learning  and  forgetting  curves  over  various  periods  of  time. 
The  LFL  curve  will  be  explained  in  more  detail  in  the  following  chapter  to 
include  graphical  and  mathematical  derivations  of  the  curve. 
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CHAPTER  2 
DEVELOPMENT  OF  CURVES 
2.1   Introduction 

Three  different  curves  will  be  developed  in  this  chapter.  Two  of  the 
curves  will  be  pseudo-learning  curves  while  the  third  will  be  a  LFL 
curve.  The  learning  curves  are  referred  to  as  pseudo-learning  curves  because 
of  the  coordinates  used  to  plot  the  curves.  A  normal  learning  curve  is 
usually  plotted  using  "Cumulative  Units"  as  the  independent  variable  and 
"Time  per  Unit"  as  the  dependent  variable,  but,  for  the  purposes  of  this 
paper,  "Calendar  Weeks"  will  be  utilized  as  the  independent  variable  and 
"Performance"  as  the  dependent  variable.  The  reasoning  behind  this  change 
in  coordinates  is  to  insure  that  the  units  of  the  results  obtained  from  the 
learning  curves  will  be  compatible  with  the  units  utilized  in  the  System 
Effectiveness  model  because,  in  Chapter  3,  it  is  required  to  have  time 
as  the  independent  variable,  and  reliability  is  expressed  over  time.  For 
the  purposes  of  this  paper,  the  units  of  the  dependent  variable,  "Performance", 
will  be  defined  in  terms  of  probability  of  success,  i.e.  reliability.  For' 
example,  a  performance  value  of  35  percent  implies  35  hits  out  of  100 
attempts  for  an  infantryman  shooting  at  a  target.  It  could  also  imply  35 
correct  observations  out  of  100  total  observations  for  a  radar  or  sonar 
operator.  The  above  definition  of  performance  will  be  explained  in  more 
detail  in  the  following  sections  of  this  chapter. 

The  first  learning  curve  is  expressed  by  a  log  equation  which  is 
similar  to  the  equation  of  the  first  learning  curve  proposed  by.  Wright  (33). 
The  second  learning  curve  has  been  formulated  in  terms  of  a  cubic  equation. 
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The  LFL  curve  is  expressed  by  three  different  equations  depending  on  the 

section  of  the  curve  under  study.  Data  used  to  develop  the  three  curves, 

(Tables  2.1,  2.3,  and  2.5),  was  obtained  from  Carlson  and  Rowe  (6). 

Carlson  and  Rowe  accumulated  this  data  by  studying  the  performance  of 

60  individuals  who  performed  the  same  skilled  manual  tasks  in  a  manufacturing 

plant. 

The  data  presented  by  Carlson  and  Rowe  (6)  was  expressed  in  terms  of 
calendar  weeks  versus  performance  where  performance  was  defined  as  the 
ratio  of  standard  time  to  actual  time.  For  the  purposes  of  this  study, 
performance  is  redefined  so  as  to  express  probability  of  success  or  reli- 
ability. Because  of  this  new  definition,  performance  means  probability  of 
success  and  not  the  usual  measure  of  quantity  output,  thus  the  value  of 
performance  must  be  less  than  or  equal  to  100%.  Because  of  this  constraint 
on  the  values  of  performance,  the  data  obtained  from  (6)  had  to  be  normalized 
because  some  of  the  performance  values  were  in  excess  of  100%.  This  trans- 
formation of  the  performance  variable  is  required  so  that  probability  of 
success  is  expressed  over  time.  This  requirement  will  become  evident  in 
Chapter  3.  Therefore,  the  basic  hypothesis  behind  the  redefinition  of  the 
performance  variable  is  that  the  probability  of  success  is  a  one-to-one 
transformation  with  the  observed  performance  data,  that  is,  it  was  assumed 
to  have  the  same  form.  Hence,  the  observed  performance  data,  as  presented 
in  (6),  was  utilized  to  generate  the  probability  of  success  data  which  was 
used  to  develop  the  learning  curves. 

The  data  for  the  log  pseudo-learning  curve  and  the  LFL  curve  was  obtained 
by  normalizing  the  original  data  presented  in  (6)  so  that  no  performance 
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values  were  in  excess  of  100  percent.     All  values  were  normalized  because 
of  the  definition  of  "Performance"  i.e.  probability  of  success   (reliability) 
can  not  exceed  a  value  of  1.00  which  is  the  same  as  a  performance  value  of 
100  percent.     The  original  data  was  utilized  to  plot  the  cubic  pseudo- 
learning  curve  because  there  were  no  performance  values  which  exceeded  100 
percent. 

The  results  from  the  three  different  curves  will  be  utilized  in  Chapter  3 
in  conjunction  with  a  model  that  formulates  System  Effectiveness.     The 
sensitivity  of  this  System  Effectiveness  model   to  the  different  curve  types 
and  to  changing  performance  values  associated  with  the  curves  will   be 
analyzed. 

2.1.1     Log  Pseudo-Learning  Curve 

This  learning  curve  is  very  similar  to  the  log-linear  learning  curve 
developed  by  Wright  in  1936  (33).     The  curve  developed  by  Wright  is  the 
simplest  and  most  easily  understood  of  all   the  learning  curves  which 
have  been  developed.     Figure  1.1   is  an  example  of  Wright's  curve  when  it 
is  plotted  using  Cartesian  coordinates.     Its  simplicity  and  ease  of  cal- 
culation make  it  the  most  widely  used  learning  curve. 

The  data  in  Table  2.1   has  been  used  to  plot  the  log  pseudo-learning 
curve  of  Figure  2.1.     Again,  this  data  is  the  result  of  normalizing  the 
original   data  presented  by  Carlson  and  Rowe  (6). 

The  model  for  the  log  pseudo-learning  curve  was  developed  using  the 
linear  regression  program  of  the  Statistical  Analysis  System  (SAS).  The 
model   has  the  following  form:     P(t)  =  31.534  +  19.549  log  t 
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where:  P(t)  =  Performance,  percent 
t  =  Time,  calendar  weeks 

The  correlation  coefficient  for  this  model  is  r  =  .997.  Table  2.2  is  the 
table  of  residuals  (residual  =  observed  value  -  predicted  value)  of  the 
model  compared  to  the  observed  values  (Table  2.1).  It  can  be  noted  in 
Table  2.2  that  the  model  gives  an  extraordinarily  good  fit  to  the  observed 
data. 

In  practical  terms,  this  type  of  learning  curve  would  result  from  a 
work  situation  where  the  individual  works  continuously  on  the  same  job, 
i.e.  he  is  not  detailed  or  assigned  to  tasks  other  than  his  main  job 
assignment.  An  example  of  this  type  of  situation  would  be  a  radar  operator 
who  does  nothing  else  except  monitor  the  radar  screen.  If  the  individual  is 
interrupted  while  working  at  his  primary  job  assignment,  this  type  of 
learning  curve  would  not  be  applicable.  Section  2.1.3  addresses  this  type 
of  interrupted  learning  experience. 

2.1.2  Cubic  Pseudo-Learning  Curve 

The  cubic  learning  curve  was  proposed  some  years  after  Wright's  log- 
linear  formulation  (33).  It  was  developed  in  an  effort  to  eliminate  the 
two  major  disadvantages  of  the  log-linear  form  i.e.  the  zero  asymptote  and 
the  assumed  constant  rate  of  learning.  This  learning  curve  plots  on 
Cartesian  coordinates  as  an  S-shaped  curve  when  "Cumulative  Units"  is  used 
as  the  independent  variable  and  "Performance"  is  used  as  the  dependent 
variable.  See  Figure  1.3  of  Chapter  1.  Many  references  exist  in  the  liter- 
ature which  address  the  theory  and  formulation  of  cubic  curves  (1),  (6) 
and  (7). 
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Week 

Performance  (%) 

1 

34.4 

2 

47.0 

3 

53.6 

4 

58.3 

5 

62.1 

6 

65.4 

7 

68.3 

S 

70.8 

9 

73.0 

10 

75.1 

11 

77.0 

12 

78.8 

13 

80.5 

14 

82.0 

15 

83.5 

TABLE  2.1 

Week 

Performance  (%) 

16 

85.0 

17 

86.4 

18 

87.6 

19 

88.9 

20 

90.1 

21 

91.2 

22 

92.3 

23 

93.4 

24 

94.4 

25 

95.4 

26 

96.4 

27 

97.3 

28 

98.2 

29 

99.1 

30 

100.0 

DATA  USED  TO  PLOT  LOG  PSEUDO-LEARNING  CURVE 
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Model:  P(t)  =  31.534  +  19.549  log  t 


Week 

Observed 
Performance  Value 

Predicted 
Performance  Value 

Residual 

1 

34.400 

31.534 

2.866 

2 

47.000 

45.084 

1.916 

3 

53.600 

53.011 

0.589 

4 

58.300 

58.634 

-0.334 

5 

62.100 

62.997 

-0.897 

6 

65.400 

66.561 

-1.161 

7 

68.300 

69.574 

-1.274 

8 

70.800 

72.185 

-1.385 

9 

73.000 

74.487 

-1.487 

10 

75.100 

76.547 

-1.447 

11 

77.000 

78.410 

-1.410 

12 

78.800 

80.111 

-1.311 

13 

80.500 

81.676 

-1.176 

14 

82.000 

83.124 

-1.124 

15 

83.500 

84.473 

-0.973 

15 

85.000 

85.735 

-0.735 

17 

86.400 

86.920 

-0.520 

18 

87.600 

88.037 

-0.437 

19 

88.900 

89.094 

-0.194 

20 

90.100 

90.097 

0.003 

21 

91.200 

91.051 

0.149 

22 

92.300 

91.960 

0.340 

23 

93.400 

92.829 

0.571 

24 

94.400 

93.661 

0.739 

25 

95.400 

94.459 

0.941 

26 

96.400 

95.226 

1.174 

27 

97.300 

95.964 

1.336 

28 

98.200 

96.675 

1.525 

29 

99.100 

97.361 

1.739 

30 

100.000 

98.023 

1.977 

TABLE  2.2 
TABLE  OF  RESIDUALS  FOR  LOG  PSEUDO-LEARNING  CURVE 
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The  data  in  Table  2.3  has  been  used  to  plot  the  cubic  pseudo-learning 
curve  of  Figure  2.2.     It  is  referred  to  as  a  pseudo-learning  curve  because 
the  coordinates  are  now  "Calendar  Weeks"  as  the  independent  variable  and 
"Performance"  as  the  dependent  variable. 

The  model  for  the  cubic  pseudo-learning  curve  was  developed  using  the 
linear  regression  program  of  the  Statistical  Analysis  System  (SAS).     The 
model  has  the  following  form: 

P(t)  =  10.622  +  12.615  t  -   .59465  t2  +   .0091986  t3 

where:     P(t)  =  Performance,  percent 

t  =  Time,  calendar  weeks 

The  correlation  coefficient  for  this  model   is  r  =  .997.     Table  2.4  is  the 
table  of  residuals  of  the  model  compared  to  the  observed  values  (Table  2.3). 
Again,  it  can  be  noted  that  the  model  gives  an  extraordinarily  good  fit 
to  the  observed  data. 

The  cubic  pseudo-learning  curve  also  would  be  obtained  in  a  work 
situation  where  the  operator  performs  only  one  task  and  is  not  interrupted 
in  his  performance  of  that  task.     An  example  of  this  type  of  continuous 
and  uninterrupted  job  position  would  be  a  telephone  operator  who  does 
nothing  else  except  work  at  a  switchboard.     The  next  section  of  this 
chapter  explains  a  job  situation  in  which  the  operator  is  interrupted 
while  performing  his  primary  duties. 

2.1.3     Learn-Forqet-Learn  (LFL)  Curve 

Little  research  has  been  done  in  the  area  of  Learn-Forget-Learn  curves. 
An  LFL  curve,  which  has  a  shape  similar  to  the  curve  in  Figure  1.4,  occurs 
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Week 

Performance  (%) 

Week 

Performance  (%) 

1 

17.7 

16 

96.3 

2 

32.6 

17 

97.1 

3 

44.8 

18 

97.8 

4 

54.7 

19 

98.4 

5 

62.8 

20 

98.8 

6 

69.4 

21 

99.1 

7 

74.8 

22 

99.4 

S 

79.3 

23 

99.6 

9 

83.0 

24 

99.7 

10 

86.1 

25 

99.8 

11 

88.7 

26 

99.8 

12 

90.8 

27 

99.8 

13 

92.6 

23 

99.7 

14 

94.0 

29 

99.6 

15 

95.3 

TABLE  2.3 

30 

99.5 

DATA  USED  TO  PLOT  CUBIC  PSEUDO-LEARNING  CURVE 
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Model : 

P(t)  =  10.622  +  12.615t  - 

.59465t2  +  .0091986t3 

Week 

Observed 
Performance  Value 

Predicted 
Performance  Value 

Residual 

1 

17.700 

22.651 

-4.951 

2 

32.600 

33.546 

-0.946 

3 

44.800 

43.362 

1.438 

4 

54.700 

52.155 

2.545 

5 

62.800 

59.979 

2.821 

6 

69.400 

66.889 

2.511 

7 

74.800 

72.941 

1.859 

8 

79.300 

78.191 

1.109 

9 

83.000 

82.692 

0.308 

10 

86.100 

86.501 

-0.401 

11 

88.700 

89.673 

-0.973 

12 

90.800 

92.263 

-1.463 

13 

92.600 

94.325 

-1.725 

14 

94.000 

95.916 

-1.916 

15 

95.300 

97.090 

-1.790 

16 

96.300 

97.902 

-1.602 

17 

97.100 

98.409 

-1.309 

IS 

97.800 

98.664 

-0.864 

19 

98.400 

98.724 

-0.324 

20 

98.800 

98.643 

0.157 

21 

99.100 

98.476 

0.624 

22 

99.400 

98.279 

1.121 

23 

99.600 

98.107 

1.493 

24 

99.700 

98.016 

1.684 

25 

99.800 

98.059 

1.741 

26 

99.800 

98.293 

1.507 

27 

99.800 

98.773 

1.027 

28 

99.700 

99.553 

0.147 

29 

99.600 

100.690 

-1.090 

30 

99.500 

102.238 

-2.738 

TABLE  2.4 
TABLE  OF  RESIDUALS  FOR  CUBIC  PSEUDO-LEARNING  CURVE 
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when  an  operator  is  interrupted  while  working  at  his  primary  job  assignment 
and  is  assigned  to  another  task,  and  then,  after  a  certain  period  of  time, 
he  returns  to  his  primary  duties.     The  curve  portrayed  in  Figure  1.4  depicts 
these  events  as  the  initial   learning  on  the  primary  task,  then  the  forgetting 
that  takes  place  during  the  interruption,  and  finally  the  resumed  learning 
of  the  task  after  the  operator  returns  to  the  job.     Instead  of  the  interruption 
taking  the  form  of  a  change  in  job  assignments,  it  could  also  indicate  a 
period  of  absence  that  the  worker  is  away  from  his  primary  job,   i.e.   a  week- 
end break  or  a  vacation  for  the  worker.     The  amount  of  forgetting  that  takes 
place  during  a  break  in  the  work  depends  on  how  much  the  worker  has  learned 
up  to  the  point  of  interruption  and  the  length  of  the  interruption  (13). 
Another  study  concerned  with  interrupted  learning  theorizes  that  a  non- 
work  interruption  (weekend  break  or  vacation)  is  not  the  same  as  a  work 
interruption  (performing  another  task)   (9).     This  theory  still  has  to  be 
verified. 

The  LFL  curve  also  can  depicit  an  individual's  increasing  performance 
during  his  initial  training  for  a  job  (first  section  of  curve),  his  de- 
creasing performance  caused  by  forgetting  since  the  initial   training 
(second  section  of  curve),  and  then  the  subsequent  increase  in  performance 
caused  by  retraining  for  the  job  (third  section  of  curve).     The  various 
work  phenomena  which  can  be  explained  by  LFL  curves  are  numerous  and  can 
be  easily  understood  by  using  this  type  of  curve.     A  practical  example  of 
a  situation  where  an  LFL  curve  could  be  applied  is  an  infantryman  who  re- 
ceived his  initial   training  on  the  use  and  firing  of  an  anti-tank  missile. 
After  the  initial   or  basic  training  period,  which  included  actual   firings 
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of  the  missile,  the  soldier  is  assigned  to  a  unit  in  which  the  actual 
firing  of  the  missile  is  impossible.     In  all   probability  he  will   forget 
some  of  the  procedures  and  techniques  required  to  fire  the  missile  during 
his  assignment  to  this  particular  unit,  and  his  performance  in  regards  to 
missile  firings  will  decrease.     In  this  particular  instance,  performance 
can  be  construed  as  accuracy  in  hitting  the  target  i.e.  probability  of 
success.     He  is  then  retrained  on  the  missile  system  by  being  assigned  to 
a  firing  range  where  he  can  perform  actual   firings  again,  and  his  performance 
level  should  increase  because  of  the  experience  he  received  on  the  range. 
See  Figure  2.3  for  a  graphical   representation  of  the  infantryman's  training 
cycle  which  was  explained  above. 

The  example  of  interrupted  learning  presented  by  Carlson  and  Rowe  (6) 
has  an  operator  performing  a  certain  task  for  a  seven  week  period,  then 
being  assigned  to  perform  another  task  for  a  period  of  12  weeks,  and  then 
returning  to  the  original   task  for  a  period  of  11  weeks.     The  performance 
data  that  portrays  the  above  sequence  of  events  is  presented  in  Table  2.5 
and  is  plotted  in  Figure  2.4.     This  data  has  again  been  normalized  from  the 
original   data  presented  in  (6)  to  insure  that  the  performance  values  do  not 
exceed  100  percent.     Again,   this  normalization  is  required  because  of  the 
definition  of  performance  which  was  explained  in  section  2.1. 

The  model   for  the  Learn-Forget-Leam  curve  was  developed  by  finding 
equations  for  each  of  the  three  sections  of  the  curve.     Each  equation  was 
formulated  using  analytical  methods.     The  model,  which  was  developed  using 
the  data  of  Table  2.5,  has  the  following  form: 


32 


1.00 


.90 


20. 


Basic 
Training 


After  Basic 
Training 


Retraining 


t       i       r 


I       I 


l       I       1        I 


I       3       5      7      9      II      13     15     17     19     21     23    25    27    29     31     33    35     37 

Calendar  Weeks 

FIGURE  2.3 
EXAMPLE  LFL  CURVE  OF  TRAINING 
CYCLE  FOR  AN  INFANTRYMAN 


33 


P(t)  - 


821.63  - 


787.23 
..02 


■15.690  + 


133.98 
..24 


Initial  Learning 


Forgetting 


108.47  -  7^8^5     ,     Resumed  Learning 


where:     P(t)  =  Performance,  percent 

t  =  Time,  calendar  weeks 

Table  2.6  is  the  table  of  residuals  of  the  model  compared  to  the  observed 
values  (Table  2.5).     It  can  be  noted  in  Table  2.6  that  the  model   gives  a 
better  than  average  fit  to  the  observed  data,  and  the  pattern  of  residuals 
bears  this  out. 

As  was  mentioned  in  the  earlier  sections  of  the  chapter,  the  LFL  curve 
can  be  utilized  when  an  individual  operator  experiences  an  interruption  in 
his  work.     The  interruption  can  take  the  form  of  a  work  interruption  (per- 
forming another  task)  or  a  non-work  interruption  (weekend  break  or  vacation), 
and  it  should  be  stated  that  the  LFL  curve  is  applicable  to  both  types  of 
interruption. 

The  results  obtained  from  the  LFL  curve  and  the  two  pseudo-learning 
curves  (log  and  cubic)  of  this  chapter's  earlier  sections  will  be  utilized 
in  conjunction  with  a  System  Effectiveness    model   in  Chapter  3.     The 
sensitivity  of  the    SE     model  to  the  three  different  curve  types  and 
changing  performance  values  of  each  curve  will  be  analyzed  and  discussed. 
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Week 

Performance  % 

1 

34.4 

2 

47.0 

3 

53.6 

4 

58.3 

5 

62.1 

6 

65.4 

7 

68.3 

S 

66.5 

9 

64.6 

10 

62.0 

11 

60.0 

12 

58.2 

13 

56.7 

14 

55.4 

15 

54.1 

Week 

Performance  (%) 

16 

53.1 

17 

52.1 

18 

51.2 

19 

50.3 

20 

56.0 

21 

60.2 

22 

63.8 

23 

66.8 

24 

69.6 

25 

71.9 

26 

74.0 

27 

76.0 

28 

77.9 

29 

79.6 

30 

81.2 

TABLE  2.5 

DATA  USED  TO  PLOT  LFL  CURVE 
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787.23 


llll    L.IQI        LCO 
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v  w  • 

-     OC  1  .1 

>o    - 

f 

0225 

Observed 

Predicted 

Week 

Performance 

Value 

Pe 

rformance  Value 

Residual 

1 

34.40 

34.40 

0 

2 

47.00 

46.58 

0.42 

3 

53.60 

53.62 

-0.02 

4 

58.30 

58.58 

-0.28 

5 

62.10 

62.40 

-0.30 

6 

65.40 

65.51 

-0.11 

7 

68.30 

68.12 

0.18 

Forgetting, 

Model:     P(t)  = 

-15. 

690  + 

133 
f 

.98 
24 

7 

68.30 

68.30 

0 

8 

66.50 

65.60 

0.90 

9 

64.60 

63.40 

1.20 

10 

62.00 

61.40 

0.60 

11 

60.00 

59.70 

0.30 

12 

58.20 

58.10 

0.10 

13 

56.70 

56.70 

0 

14 

55.40 

55.40 

0 

15 

54.10 

54.30 

-0.20 

16 

53.10 

53.20 

-0.10 

17 

52.10 

52.20 

-0.10 

18 

51.20 

51.30 

-0.10 

19 

50.40 

TABLE 

2.( 

50.40 

0 

TABLE  OF  RESIDUALS  FOR  LFL  CURVE 
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Week 

Observed 
Performance  Value 

Predicted 
Performance  Value 

Residual 

19 

50-40 

50.40 

0 

20 

56.00 

55.10 

0.90 

21 

60.20 

59.20 

1.00 

22 

63.80 

62.90 

0.90 

23 

66.80 

66.10 

0.70 

24 

69.60 

69.00 

0.60 

25 

71.90 

71.60 

0.30 

26 

74.00 

73.90 

0.10 

27 

76.00 

76.00 

0 

28 

77.90 

77.80 

0.10 

29 

79.60 

79.60 

0 

30 

81.20 

81.10 

0.10 

TABLE  2.6     continued 
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CHAPTER  3 
SYSTEM  EFFECTIVENESS  MODELS 
3.1  Introduction 

Before  proceeding  to  the  discussions  of  this  chapter,  the  terra, 
"System  Effectiveness"  should  be  defined.  Gephart  and  Balachandran  (10) 
define  it  as:  "The  probability  that  the  man-machine  system  will  success- 
fully meet  an  operational  demand  and  fulfill  the  predetermined  mission  ob- 
jectives within  a  given  mission  time  when  operated  under  stated  conditions." 
In  language  that  is  easier  to  understand,  System  Effectiveness  is  "the 
probability  that  a  system  can  successfully  meet  an  operational  demand 
throughout  a  given  time  period  when  operated  under  specified  conditions." 
In  most  cases,  System  Effectiveness  is  stated  in  probabilistic  form, 
i.e.  probability  of  system  success. 

This  chapter  is  organized  into  two  major  sections.  The  first  section 
consists  of  a  summary  of  three  different  System  Effectiveness  models.  These 
three  models  will  be  identified  as  (1)  The  Modified  WSEIAC  Model,  (2)  The 
Navy  Model,  and  (3)  Lie's  Model.  After  the  summary,  a  comparison  will  be 
made  between  the  three  models  with  differences  and  similarities  being  dis- 
cussed. 

The  summary  of  Lie's  model  will  be  in  more  detail  than  the  other  two 
models  because  Lie's  formulation  will  form  the  basis  for  the  next  major 
section  of  this  chapter.  Lie's  proposed  model,  is  very  similar  in  some 
aspects,  to  the  other  two  models,  but  it  addresses  two  areas  (environmental 
and  operator  impact  on  SE  )  which  were  not  mentioned  or  only  briefly 
explained  in  the  first  two  models.  The  area  of  operator  impact  on  System 
Effectiveness  is  of  major  interest  in  this  chapter. 
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The  performance  values  obtained  from  the  two  pseudo-learning  curves 
(log  and  cubic)  and  the  Learn-Forget-Learn  curve  of  Chapter  2  will  be  used 
in  conjunction  with  Lie's  model  to  analyze  the  effect  that  the  various 
curve  forms  and  associated  levels  of  performance  have  on  the  value  of 
System  Effectiveness.  These  changes  in  the  values  of  System  Effectiveness 
then  will  be  analyzed  and  discussed  in  Chapter  4.  Also,  an  analysis  of  the 
behavior  of  the  LFL  curve  will  be  undertaken  in  the  last  section  of  this 
chapter. 

3.2  Comparison  of  System  Effectiveness  Models 

As  was  mentioned  in  the  preceding  section,  three  proposed  models  that 
attempt  to  quantify  System  Effectiveness  will  be  summarized  and  compared 
in  this  section  of  the  chapter.  The  three  models  will  be  referred  to  as 
(1)  The  Modified  WSEIAC  Model,  (2)  The  Navy  Model,  and  (3)  Lie's  Model. 
The  terms  used  in  the  discussion  and  analysis  are  defined  as: 

(1)  Availability  -  The  probability  that  the  system  is  in  an  "up" 
and  ready  state  at  the  beginning  of  the  mission  when  the  mission  occurs  at 
a  random  point  in  time.  Availability  is  a  function  of  the  reliability  and 
maintainability  characteristics  of  the  system. 

(2)  Reliability  -  The  probability  that  an  item  will  perform  its  in- 
tended function  for  a  specified  interval  under  stated  conditions. 

(3)  Maintainability  -  The  probability  that  an  item  will  be  retained 
in  or  restored  to  a  specified  condition  within  a  given  period  of  time. 

(4)  Dependability  -  The  probability  that,  given  the  system  was 
available,  it  will  continue  to  operate  throughout  the  mission  either  (1) 
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without  a  system-level  failure  (a  failure  that  causes  the  entire  system 
to  be  inoperable),  or  (2)  if  it  fails,  it  will  be  restored  to  operation  within 
some  critical  time  interval  which,  if  exceeded,  would  result  in  mission 
failure.  Dependability  is  also  a  function  of  the  reliability  and  maintain- 
ability characteristics  of  the  system. 

(5)  Capability  -  The  probability  that  the  system's  designed  performance 
will  allow  it  to  meet  mission  demands  successfully  assuming  that  the  system 
is  available  and  dependable. 

Now  that  some  of  the  more  important  terms  have  been  defined,  we  can 
proceed  to  the  summaries  of  the  three  models. 

3.2.1  The  Modified  WSEIAC  Model 


In  1963,  the  Weapon  System  Effectiveness  Industry  Advisory  Committee 
(WSEIAC)  was  formed  for  the  purpose  of  "providing  technical  guidance  and 
assistance  to  the  Commander,  Air  Force  Systems  Command,  in  the  development 
of  a  technique  to  appraise  management  of  current  and  predicted  System 
Effectiveness  at  all  phases  of  system  life."  (10).  The  committee  theorized 
that  System  Effectiveness  was  a  joint  probability  measure  expressed  as: 

SE  =  (A)(0)(C)  (1) 

where:    SE  =  System  Effectiveness 

A  =  Availability  of  the  system 

D  =  Dependability  of  the  system 

C  =  Capability  of  the  system 
and  where  A,  D,  and  C  are  probability  statements. 
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They  further  stated  that  availability  (A)  may  be  obtained  as  a  function 
of  the  state  readiness  of  the  system  and  the  utilization  of  the  system. 
This  meant  that  equation  (1)  was  then  transformed  to  be  the  following 
expression: 

SE  =  (V)(W)(D)(C)  (2) 

where:      V  =  a  measure  of  state  readiness  of  the  system. 

W  =  a  measure  of  the  probability  of  utilizing  the 
system  given  the  state  of  the  system, 
and  where  V  and  W  are  probability  statements. 
In  1969,  Gephart  and  Balachandran  modified  equation  (2)  by  making 
the  following  changes: 

(1)  They  relabelled  V  to  become  S  (state  readiness),  and  relabelled 
W  to  become  U  (utility). 

(2)  They  relabeled  D  as  RE-RE  (Reliability-Repairability). 

(3)  And  lastly,  they  proposed  that  the  capability  term  of  equation 
(2),  C,  could  be  expressed  as  the  product  of  "adequacy  of  personnel",  A, 
and  "capability  of  hardware",  CH. 

With  the  above  changes  being  made  to  equation  (2),  it  would  then  take 
the  following  form: 

SE  =  (S)(U)(RE-RE)(A)(CH)  (3) 

where    S,  U,  RE-RE,  A,  and  CH  are  probability  statements. 

The  major  modification  that  Gephart  and  Balachandran  made  to  the 
original  WSEIAC  model,  equation  (2),  was  to  partition  the  capability  of 
the  system,  C,  into:  (a)  that  which  was  contributed  by  the  hardware  of 
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the  system,  and  (b)  that  which  was  attributable  to  the  human  factor  (oper- 
ator). By  partitioning  the  capability  of  the  system,  they  realized  that 
the  performance  of  the  operator  has  a  definite  impact  on  System  Effective- 
ness. In  earlier  studies,  operator  performance  was  assumed  to  have  a  con- 
stant value  of  100  per  cent  i.e.,  the  operator  was  totally  reliable.  In 
actuality,  an  operator's  performance  is  very  seldom  totally  reliable  and 
therefore,  this  assumption  led  to  miscalculations  of  System  Effectiveness. 

They  defined  "adequacy  of  personnel"  as  the  conditional  probability 
that  the  personnel  will  perform  at  their  level  of  proficiency,  given  that 
the  hardware  component  of  the  system  is  in  a  given  state.  They  assumed 
in  the  model  that  the  variable  which  describes  operator  performance  follows 
a  normal  distribution.  They  further  stated  that  the  parameters  of  this  dis- 
tribution can  be  obtained  from  the  training  programs  or  proficiency  evalu- 
ations of  a  sample  from  the  relevant  population  of  subjects.  These  para- 
meters then  can  be  used  in  the  Systems  Effectiveness  simulation. 

However,  they  did  not  detail  how  to  obtain  useable  human  performance 
data  from  the  training  programs  or  proficiency  evaluations.  In  other  words, 
they  presented  no  analytical  method  which  could  be  used  to  extract  data 
from  the  training  programs/proficiency  evaluations.  Without  being  able  to 
extract  human  performance  data  from  the  sources  they  mentioned,  the  human 
performance  portion  of  their  Capability  term  is  useless.  Because  of  this, 
no  operator  performance  data  was  utilized  in  the  example  problem  they  pre- 
sented in  their  paper.  Therefore,  in  essence,  Gephart  and  Balachandran 
outlined  the  requirement  for  including  an  operator  performance  term  in  the 
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calculation  of  System  Effectiveness,  but  did  not  explain  how  to  obtain 
operator  performance  data  which  could  then  be  utilized  in  the  System 
Effectiveness  simulation. 

3.2.2  The  Navy  Model 

The  Navy  Model  for  System  Effectiveness  was  obtained  from  a  proposed 
revision  of  the  "Navy  System  Effectiveness  Manual"  which  was  written  by 
D.  T.  Hanifan  (14).  This  model  is  very  similar  to  the  Modified  WSEIAC 
Model  presented  in  the  last  section.  In  the  manual,  Hanifan  states  that 
"the  effectiveness  of  a  system  depends  on  its  availability,  dependability, 
and  capability  in  relation  to  the  mission."  This  statement  expresses  the 
same  formulation  for  System  Effectiveness  as  was  presented  by  equation  (1) 
of  the  last  section,  i.e.: 

SE-(A)(D)(C)  (1) 

Hanifan  goes  on  to  state  that  the  three  terms  of  the  model  are  mutually 
exclusive,  and  great  care  should  be  exercised  in  modeling  to  guarantee  that 
the  same  data  are  not  included  in  more  than  one  term  of  the  model. 

As  was  the  case  for  the  Modified  WSEIAC  Model,  Hanifan  says  that  the 
"Capability"  term,  C,  of  equation  (1)  can  be  partitioned  into  a  term 
which  is  contributed  by  the  hardware  of  the  system  and  a  term  which  is 
attributable  to  the  performance  of  the  operator.  Usually,  Capability  is 
less  than  theoretical  computations  or  test  results  because  the  human  per- 
formance part  of  the  Capability  term  may  have  been  overestimated  or  even 
assumed  to  be  1.0  (which  is  the  assumption  when  the  human  performance  term 
is  effectively  left  out).  Hanifan  says  that  because  of  the  above  assumption, 
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"the  effectiveness  modeler  must  usually  modify  the  system  performance  numbers 
obtained  from  hardware  designers  in  order  to  obtain  a  more  accurate  estimate 
of  total  system  Capability."  Because  of  the  difficulty  in  obtaining  suitable 
human  performance  data,  estimates  for  this  data  must  frequently  be  substi- 
tuted for  empirically-obtained  data.  He  says  some  of  the  human  performance 
parameters  can  be  estimated  from  experimental  data  or  operational  records, 
but  many  are  at  present  known  only  qualitatively  and  their  effects  must  be 
estimated  on  the  basis  of  judgement.  Too  often  the  tendency  is  to  leave 
the  human  performance  data  completely  out  of  the  model. 

As  in  the  last  section,  the  author  notes  the  importance  of  including 
an  operator  performance  term  in  the  formulation  of  System  Effectiveness, 
but  gives  no  concrete  method  for  obtaining  data  which  can  be  used  in  the 
operator  performance  term.  In  addition  to  not  presenting  any  concrete 
method  for  obtaining  this  data,  he  does  not  present  a  method  for  estimating 
the  data  that  would  be  required  to  formulate  the  human  performance  term. 
He  discusses  the  importance  of  human  performance,  but  that's  all. 

3.2.3  Lie's  Model 


The  formulation  of  this  model  is  contained  in  Lie's  doctorial  disser- 
tation (16).  Lie  developed  numerous  models  that  attempted  to  quantify 
Mission  Effectiveness  (  ME  ).  To  be  consistent  with  the  terminology  used 
in  the  preceding  two  models,  we  note  that  System  Effectiveness  is  Mission 
Effectiveness,  hence  Lie's  term  of  Mission  Effectiveness  will  be  labelled 
System  Effectiveness. 
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Lie's  general   formulation  of  System  Effectiveness  is: 

<SE>io  ■  nY'iV'M'Vu  (4) 

where:  (SE)..  =  System  Effectiveness  of  unit  i  for  mission  j 

A1?.  =  Availability  of  the  hardware  component  of  unit  i 

at  the  start  of  mission  j 
A?.  =  Availability  of  the  operator  component  of  unit  i 

I  J 

at  the  start  of  mission  j 
R1?.  =  Mission  reliability  of  the  hardware  component  of 

unit  i  for  mission  j 
R?.  =  Mission  reliability  of  the  operator  of  unit  i  for 

mission  j 
E--  =  Performance  of  unit  1  during  mission  j  for  a  given 

status  of  the  environment 
P.  •  =  Performance  of  the  operator  of  unit  i  during 

mission  j 
and  where  all  the  terms  of  equation  (4)  are  probability  statements. 

Equation  (4)  is  comparable  to  equation  (1)  of  the  preceding  sections  except 
for  one  major  deviation  -  the  "Capability"  portion  of  the  model  represented 
by  equation  (1)  is  now  expressed  by  a  term  for  environmental  impact  on  SE 
and  a  term  that  deals  with  operator  performance  and  its  effect  on  SE. 
Lie  states  that  "the  performance  of  a  unit  is  dependent  upon  the 
status  of  the  environment",  i.e.  a  better  performance  of  the  unit  is 
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expected  in  a  good  weather  condition  than  in  a  bad  weather  condition 
(cold  winter,  stormy  night,  etc.).  Lie  classified  the  status  of  the  en- 
vironment as  excellent,  good,  fair,  poor,  etc.  He  also  expressed  the  idea 
that  the  performance  of  a  unit  is  dependent  upon  the  performance  of  the 
operator,  and  performance  is  a  function  of  the  quality  of  the  operator  and 
the  retraining  period.  He  assumed  that  the  performance  of  the  operator  of 
unit  i  can  be  expressed  by  the  following  functional  form: 

*i  =  ^1>1  +  &2>1  e"ei  {5) 

where:  y-   =  Probability  of  the  mission  success  as  a  function  of 
the  operator  effect  of  unit  i  at  time  t 
(y-i)-j  =  Steady-state  probability  of  the  mission  success  as  a 
function  of  the  operator  effect  of  unit  i 
(yi)i+(y2)i  =  Initial  peak  probability  of  the  mission  success  as  a 
function  of  the  operator  effect  of  unit  i 
6.  =  Decreasing  rate  of  the  probability  of  the  mission 

success  as  a  function  of  the  operator  effect  of  unit  i 
t  =  time,  hours 
If  the  retraining  of  the  operator  is  performed  every  Ti  time  units,  and  if 
every   retraining  brings  the  performance  up  to  the  initial  level,  then  the 
performance  of  the  operator  of  unit  i  may  be  represented  as  shown  in 
Figure  3.1 . 

Lie  states  that  equation  (5)  is  one  of  a  variety  of  functional  forms 
for  the  operator  performance  that  can  be  assumed.  Equation  (5)  is  an 
expression  for  operator  performance  and  could  be  termed  an  "LFL"  curve. 
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FIGURE  3.1 
PERFORMANCE  OF  THE  OPERATOR  OF  UNIT  i  AS  A 
FUNCTION  OF  THE  QUALITY  OF  THE  OPERATOR 
((y^-.  (y2)r  and  (3.)  AND  THE  RETRAINING 
PERIOD  (T^). 
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Lie  assigned  arbitrary  values  to  the  parameters  of  equation  (5)' when  the 
expression  was  utilized  in  his  simulation  of  the  model.  He  provided  no 
explanation  as  to  why  he  used  the  particular  equation  he  did,  equation  (5), 
or  why  he  chose  the  particular  values  for  the  parameters  that  he  did.  He 
did  not  mention  how  values  for  the  parameters  could  be  obtained  from 
training  programs,  proficiency  evaluations,  etc. 

Using  the  equations  for  operator  performance  developed  in  Chapter  2 
from  the  learning  and  LFL  curves  in  place  of  equation  (5),  the  simulation 
program  for  System  Effectiveness  (the  simulation  program  for  SE  is  listed 
in  Appendix  A) which  Lie  developed  was  run  to  determine  what  effect  the 
various  curve  forms  would  have  on  the  overall  System  Effectiveness.  The 
results  of  the  various  simulation  runs  will  be  outlined  in  section  3.3.  with 
subsequent  conclusions  made  in  Chapter  4. 

3.2.4  Comparison  of  Models 

It  should  be  evident  from  the  three  preceding  sections  that  the  three 
models  are  very  similar  in  most  aspects  with  the  only  differences  being  in 
the  interpretation  of  the  "Capability"  term  of  equation  (1).  The  Modified 
WSEIAC  Model  and  the  Navy  Model  are  almost  identical  in  their  formulation  of 
System  Effectiveness  while  the  model  of  Lie's  differs  in  the  make-up  of 
the  "Capability"  term.  Lie  also  developed  his  model  for  Mission  Effective- 
ness while  the  other  two  were  formulated  in  terms  of  System  Effectiveness, 
but  both  terms  (  SE  and  ME  )  employ  the  same  concepts. 

The  one  fact  that  should  be  brought  out  in  this  comparison  is  that  none 
of  the  models  provided  any  definitive  data  on  human  performance.  Also,  no 
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guidance  at  all  was  provided  in  regards  to  obtaining  analytical  expressions 
that  could  be  used  to  quantify  human  performance.  Lie  did  the  most  work 
in  this  area,  but  he  was  deficient  in  the  explanation  of  the  equation  he 
used,  and  he  did  not  list  possible  sources  of  human  performance  data.  As 
far  as  the  operator  performance  term  in  each  of  the  models,  there  was  much 
discussion  about  what  should  be  done  in  this  area,  but  no  one  gave  any 
direction  that  could  be  followed  when  trying  to  quantify  human  performance 
levels.  This  paper  will  give  insight  into  the  collecting  of  human  per- 
formance data, development  of  analytical  expressions  for  the  performance, 
and  their  subsequent  use  in  effectiveness  simulation  models. 

3.3  Lie's  Simulation  Model  with  Modified  Operator  Performance  Term 

Lie  (16)  developed  a  number  of  simulation  models  that  attempted  to 

quantify  System  Effectiveness.  The  models  varied  according  to  the  constraints 

and  assumptions  that  were  applied  to  the  various  systems.  The  particular 

simulation  model  that  will  be  utilized  in  this  section  was  developed  for 

a  system  which  was  required  to  carry  out  various  types  of  missions.  In 

this  particular  model,  each  mission  type  is  characterized  by  the  maximum 

allowable  time  that  determines  the  success  of  a  given  mission  type.  Lie 

described  the  logic  of  this  model  in  the  following  way: 

"For  a  given  type  of  a  mission  to  be  successful,  the  system  is  re- 
quired to  be  available  at  the  start  of  a  mission,  and  the  system 
must  complete  its  mission  within  the  maximum  allowable  duration 
of  time  that  this  given  mission  type  specifies  without  any  failure 
during  this  period.  If  the  system  cannot  accomplish  a  mission 
within  the  specified  duration  of  time,  the  mission  is  terminated 
at  this  point  and  is  considered  to  be  failed  even  though  the  system 
is  still  operable.  Failures  of  the  system  are  induced  by  both  the 
hardware  itself  and  the  operator.  Furthermore,  the  effects  of  the 
environment  and  the  operator  are  reflected  in  the  mission  duration. 
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In  other  words,  poor  environmental  conditions  and  poor  operator 
performance  are  assumed  to  make  the  actual  mission  duration 
longer  than  the  mission  duration  under  ideal  conditions.  Thereby, 
adverse  effects  of  the  environment  and  the  operator  tend  to  re- 
duce the  probability  of  mission  success,  i.e.  System  Effectiveness. 

Hopefully,  this  short  synopsis  of  the  system  will  help  explain  the  simu- 
lation model  for  this  particular  system.  Again,  a  printout  of  the  simu- 
lation program  used  in  this  section  is  listed  in  Appendix  A. 

The  section  of  the  simulation  program  which  was  of  major  interest 
in  this  paper  dealt  with  the  operator  performance  term,  0P(I,J),  and  its 
formulation.  In  the  simulation  program  (Appendix  A),  cards  number  177 
through  206  calculated  the  operator  performance  for  unit  i  and  mission  j, 
0P(I,J),  and  printed  the  various  values  of  0P(I,J)  in  the  output. 
Equation  (5)  of  section  3.2.3  was  used  by  Lie  to  express  the  operator  per- 
formance of  unit  i  and  mission  j  in  the  simulation  program.  He  assumed 
the  following  values  for  the  parameters  of  equation  (5): 

y]  =  .8,y2  =  .2,  3  =  -0014,  T  =  2160  hours 

where  y-, ,  y2>  and  6  are  probability  values. 
When  the  above  values  were  used  in  equation  (5),  and  a  total  of  50  missions 
were  simulated  for  a  single  unit,  the  operator  performance  for  each  of  the 
50  missions  was  calculated  to  be  the  values  in  Table  3.3.  Using  the  values 
of  Table  3.3,  the  overall  System  Effectiveness  for  the  unit  turned  out  to 
be  .52  after  all  the  calculations  of  the  simulation  were  completed. 

The  equations  for  operator  performance,  which  were  developed  in 
Chapter  2  from  the  three  different  curves,  were  substituted  into  Lie's 
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simulation  program  in  place  of  his  expression  for  0P(I,J)  to  determine 
what  values  of  operator  performance,  and  subsequently,  what  value  of 
System  Effectiveness  would  result.     The  equations  from  Chapter  2  are  as 
follows: 

Log  Pseudo-Learning  Curve 
0P(I,J)  =  31.534  +  19.549  log  t 

Cubic  Pseudo-Learning  Curve 
0P(I,J)  =  10.622  +  12.615  t  -  0.59465  t2  +  0.0091986  t3 

Learn-Forget-Learn  Curve 


Initial  Learning,  0P(I,J)  =  821.63  -  78q^23 


Forgetting,  0P(I,J)  =  -15.690  +  13J^98 


Resumed  Learning,  0P(I,J)  =  108.47  -  ^Js 

Note  that  "t"  in  the  above  equations  was  replaced  by  "CTMS(I,J)"  when  the 
equations  were  utilized  in  the  simulation  program.     "CTMS(I,J)"  stands 
for  the  mission  start  time  expressed  in  clock  time  for  unit  i  and  mission  j, 
This  transformation  was  made  because  the  simulation  program  operates  on 
a  clock  time  basis,   and  "CTMS(I.J)"  is  Lie's  clock  time  term  which  is 
equivalent  to  "t". 

Each  of  the  three  equations  were  substituted  into  Lie's  program  with 
each  equation  being  run  separately  in  the  simulation.     The  resulting  values 
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of  operator  performance  for  the  50  missions  obtained  from  the  Log  Pseudo- 
Learning  curve,  Cubic  Pseudo-Learning  curve,   and  Learn-Forget-Learn 
curve  are  listed  in  Tables  3.4,  3.5,  and  3.6  respectively.     The  subsequent 
values  for  overall  System  Effectiveness  after  the  three  equations  from 
Chapter  2  and  Lie's  equation  were  utilized  are  tabulated  in  Table  3.7. 
The  values  of  0P(I,J)  =   .50  and  1.0  were  also  used  in  the  simulation  to 
obtain  a  range  for  the  System  Effectiveness  values.     The  resulting   SE 
values  for  these  two  constant  operator  performance  terms  are  also  listed 
in  Table  3.7. 

3.4    Analysis  of  Learn-Forget-Learn   (LFL)  Curve 

The  mean  performance  value  for  the  Learn-Forget-Learn  curve,  Figure  2.4, 
is  calculated  in  this  section  along  with  the  corresponding  mean' value  of 
System  Effectiveness  which  results  when  the  performance  mean  is  utilized 
in  Lie's  simulation  program.     Also,  in  this  section,  the  average  operator 
performance  is  calculated  for  sample  missions  taken  from  the  total  of  50 
missions.     The  values  of  the  above  mean  performance  figures  and  their  re- 
sulting System  Effectiveness  indexes  will  be  compared  and  discussed  in 
Chapter  4. 

3.4.1  Mean  Performance  of  LFL  Curve 

The  mean  performance  value  for  the  Learn-Forget-Learn  curve  is  cal- 
culated by  integrating  the  three  separate  segments  of  the  curve  over  the 
time  periods  that  they  cover  and  then  dividing  the  sum  of  the  integration 
results  by  the  total   time  period  for  which  the  curve  is  effective.     The  LFL 
curve  is  portrayed  in  Figure  2.4  with  the  equation  for  each  of  the  three 
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sections  of  the  curve  listed  above  the  portion  of  the  curve  to  which  it 
applies.     The  integration  which  yields  the  area  below  the  LFL  curve  is  as 
follows: 


/  (821.63  -  2§Zi?i)dt  +  /     (-15,690  +  H2j2&)dt  +  /     (108.47  -  Zr%)dt 
1  f0^  7  t 4  19  t'-D0 

From  the  above  integration,  the  total   area  below  the  curve  is  found  to  be: 


339.18  +  690.41   +  757.13  =  1786.72  weeks 


When  this  total  area  is  divided  by  the  total  time  interval  for  which  the 
curve  is  effective,  the  mean  operator  performance,  O.P.,  will   result: 


0.P, 


1786.12  _ 
29 


61.6% 


By  using  the  predicted  performance  values,   P^ ,  from  the  table  of 
residuals  for  the  LFL  curve,  Table  2.6,   in  conjunction  with  the  value  of 
P,  the  standard  deviation  that  pertains  to  the  mean  performance,  s         ,  can 


0.P, 


be  calculated  as  follows 


30 


O.P. 


I     (PrP)' 
i=l       n 


n-1 


1/2 


3197.99 


30-1 


1/2 


=  10.50% 


The  values  of  O.P.   and  s will   be  utilized  in  Chapter  4  for  comparing 

O.P. 

various  values  obtained  from  the  LFL  curve.  Before  proceeding  to  the  next 
section,  it  should  be  noted  that  the  value  of  P  obtained  above  is  only  valid 
for  a  large  number  of  missions,  and  it  should  not  be  utilized  when  estimating 
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the  performance  for  a  single  mission  or  a  small  sample  of  missions. 
The  reasoning  behind  this  statement  will  be  demonstrated  in  Chapter  4. 

When  the  value  of  07P.  is  substituted  into  Lie's  simulation  program, 
the  System  Effectiveness  turns  out  to  be  .38. 

3.4.2  Mission  Sampling 

To  study  the  sensitivity  of  the  operator  performance  at  a  point  in 
time  versus  the  overall  performance  value,  four  samples  (n=4),  each  of 
size  five,  of  missions  out  of  the  total  of  50  simulated  mission  were  taken. 
The  four  samples  were  taken  from  around  the  10th,  20th,  30th,  and  40th 
mission  intervals.  As  was  mentioned  earlier,  each  sample  will  consist 
of  five  observations.  For  example,  the  sample  for  the  10   mission  interval 
is  comprised  of  five  observations,  i.e.  readings  from  mission  numbers  8,  9, 
10,  11,  and  12.  From  the  computer  output  of  Lie's  simulation  which  was 
run  using  the  equations  for  the  LFL  curve,  operator  performance  values 
(O.P.)  for  each  individual  mission  are  obtained.  The  average  operator  per- 
formance for  each  of  the  four  samples  is  then  calculated  as  depicted  in 
Table  3.1. 

When  each  of  the  0TF...  (i  =  10,20,30,40)  are  averaged  together,  the 


resulting  value,  the  grand  mean  (O.P.)  of  the  four  samples,  turns  out  to  be: 


O.P.  =  60% 

Note  that  the  value  of  60%  obtained  in  this  section  is  very  close  to  the 
value  of  62%  obtained  in  section  3.4.1.  This  result  is  only  logical  because 
the  two  mean  values  were  obtained  from  the  same  population.  The  small 
difference  in  their  values  is  the  result  of  the  two  different  methods 
utilized  to  calculate  the  means. 
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By  using  the  values  of  O.P..   ( i=l 0,20,30,40)  in  conjunction  with  the 
value  of  O.P.,  the  standard  deviation  that  pertains  to  the  mean  performance 
calculated  in  this  section  is  found  as  follows: 


s  = 


4 

2] 

'/  c 

O.P.-O.P. 

• 

n-1 

. 

162 

4-1 


1/2 


7.35% 


The  values  of  O.P.  and  s  which  were  calculated  in  this  section  will  be 
utilized  in  Chapter  4  for  comparing  various  values  obtained  from  the  LFL 
curve. 

For  each  of  the  four  mission  intervals  sampled,  the  average  System 
Effectiveness  (  SE  )  for  each  interval  can  be  calculated  by  using  the 
computer  output  of  Lie's  simulation.  It  should  be  noted  that  System 
Effectiveness  can  only  have  values  of  0  or  1 ,  i.e.  the  mission  either 
fails  or  it  is  successful.  The  calculations  for  average  System  Effectiveness 
are  depicted  in  Table  3.2. 

When  each  of  the   ST.  (i=l 0,20,30,40)  are  averaged  together,  the 
resulting  value,  SE  ,  turns  out  to  be: 

ST  =  .40 
Note  that  the  average  value  for  System  Effectiveness  obtained  in  this 
section,  .40,  and  for  the  entire  period  found  in  the  preceding  section, 
.38,  are  very  close,  as  well  they  should  be,  because  they  were  obtained 
from  the  same  population  of  values.  The  small  difference  in  their  values 
results  from  the  two  different  methods  utilized  to  calculate  the  averages. 
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One  final   item  should  be  mentioned  in  this  section.     Since  the  results 
of  this  section  were  calculated  by  using  the  output  from  Lie's  simulation 
and  since  the  form/content  of  the  output  was  not  listed  here,     Lie's  dis- 
sertation (16)  can  be  consulted  for  further  explanation. 
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J.L. 

10   Mission  Interval 


20   Mission  Interval 


Mission  No. 

O.P. 

8 

61 

9 

61 

10 

59 

11 

58 

12 

n=59 

57 

0. 

P.-. 

Mission  No. 

O.P. 

18 

55 

19 

54 

20 

54 

21 

53 

22 

52 

0, 

,V*f 

=54 

30   Mission  Interval 


+■  h 

40   Mission  Interval 


Mission 

No. 

O.P. 

Mi 

ssion  No. 

O.P. 

28 

50 

38 

67 

29 

52 

39 

69 

30 

56 

40 

70 

31 

58 

41 

71 

32 

'30= 

=55 

60 

42 

'40= 

=70 

72 

0. 

P, 

0. 

P 

TABLE  3.1 


CALCULATIONS  OF  O.P.. 
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10th  Mission  Interval 


20th  Mission  Interval 


Mission  No. 

SE 

8 

0 

9 

0 

10 

1 

11 

0 

12 

SE1(f.20 

0 

Mission 

No. 

SE 

18 

1 

19 

1 

20 

0 

21 

1 

22 

SE20= 

.60 

0 

+■  h 

30   Mission  Interval 


J.L. 

40   Mission  Interval 


Mission  No. 

SE 

28 

0 

29 

0 

30 

0 

31 

0 

32 

SE30=0 

0 

Mission  No. 

SE 

38 

1 

39  - 

0 

40 

1 

41 

1 

42 

SE4Q=.80 

1 

TABLE  3.2 


CALCULATIONS  OF  SE, 
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Mission  No. 

Operator  Performance 

1 

.99 

2 

.99 

3 

.98 

4 

.98 

5 

.97 

6 

.97 

7 

.97 

8 

.97 

9 

.97 

10 

.97 

11 

.96 

12 

.96 

13 

.96 

14 

.96 

15 

.96 

16 

.96 

17 

.96 

18 

.96 

19 

.96 

20 

.95 

21 

.95 

22 

.95 

23 

.95 

24 

.95 

25 

.95 

Mission  No. 

Operator  Performance 

26 

.95 

27 

.95 

28 

.95 

29 

.94 

30 

.94 

31 

.94 

32 

.94 

33 

.94 

34 

.94 

35 

.94 

36 

.94 

37 

.94 

38 

.94 

39 

.93 

40 

.93 

41 

.93 

42 

.93 

43 

.93 

44 

.93 

45 

.93 

46 

.93 

47 

.93 

48 

.93 

49 

.93 

50 

.92 

TABLE  3.3 


OPERATOR  PERFORMANCE  VALUES  RESULTING 
FROM  LIE'S  EQUATION 
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Mission  No. 

Operator  Performance 
.45 

Mission  No.    0 
26 

perator  Performance 

1 

.88 

2 

.59 

27 

.89 

3 

.67 

28 

.89 

4 

.72 

29 

.89 

5 

.74 

30 

.90 

6 

.74 

31 

.91 

7 

.76 

32 

.91 

8 

.77 

33 

.91 

9 

.77 

34 

.92 

10 

.79 

35 

.92 

11 

.80 

36 

.92 

12 

.81 

37 

.93 

13 

.82 

38 

.93 

14 

.82 

39 

.94 

15 

.82 

40 

.94 

16 

.83 

41 

.94 

17 

.83 

42 

.95 

18 

.83 

43 

.95 

19 

.84 

44 

.95 

20 

.85 

45 

.95 

21 

.86 

46 

.96 

22 

.87 

47 

.96 

23 

.87 

48 

.96 

24 

.87 

49 

.96 

25 

.88 

50 

.98 

TABLE  3.4 

OPERATOR  PERFORMANCE  VALUES  RESULTING 
FROM  LOG  PSEUDO-LEARNING  EQUATION 
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Mission  No. 

Operator  Performance 

Mission  No. 

Operator  Performance 

1 

.34 

26 

.99 

2 

.52 

27 

.99 

3 

.67 

28 

.99 

4 

.78 

29 

.99 

5 

.81 

30 

.99 

6 

.82 

31 

.99 

7 

.85 

32 

.98 

8 

.87 

33 

.98 

9 

.88 

34 

.98 

10 

.91 

35 

.98 

11 

.92 

36 

.98 

12 

.94 

37 

.98 

13 

.94 

38 

.98 

14 

.95 

39 

.98 

15 

.95 

40 

.98 

16 

.95 

41 

.98 

17 

.96 

42 

.98 

18 

.96 

43 

.98 

19 

.97 

44 

.98 

20 

.98 

45 

.98 

21 

.98 

46 

.99 

22 

.98 

47 

.99 

23 

.98 

48 

.99 

24 

.99 

49 

.99 

25  .99  50  1.00 

TABLE  3.5 

OPERATOR  PERFORMANCE  VALUES  RESULTING 
FROM  CUBIC  PSEUDO-LEARNING  EQUATION 
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Mission  No. 

Operator  Performance 

Mission  Nn. 

Oppratnr  Pprfnrmanro 

1 

.47 

26 

.51 

2 

.59 

27 

.51 

3 

.66 

28 

.50 

4 

.66 

29 

.52 

5 

.64 

30 

.56 

6 

.64 

31 

.58 

7 

.62 

32 

.60 

8 

.61 

33 

.61 

9 

.61 

34 

.62 

10 

.59 

35 

.63 

11 

.58 

36 

.65 

12 

.57 

37 

.66 

13 

.57 

38 

.67 

14 

.56 

39 

.69 

15 

.56 

40 

.70 

16 

.56 

41 

.71 

17 

.55 

42 

.72 

18 

.55 

43 

.73 

19 

.54 

44 

.73 

20 

.54 

45 

.74 

21 

.53 

46 

.75 

22 

.52 

47 

.75 

23 

.52 

48 

.77 

24 

.52 

49 

.77 

25 

.52 

TABLE  3.6 

50 

.81 

OPERATOR  PERFORMANCE  VALUES  RESULTING 
FROM  LEARN-FORGET-LEARN  EQUATION 
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Operator  Performance  Equation 

System 

Effectiveness 

Lie's 

.52 

Log  Pseudo-Learning 

.56 

Cubic  Pseudo-Learing 

.58 

Learn-Forget-Learn  (LFL) 

.42 

Constant  value  of  .50 

.32 

Constant  value  of  1.0 

TABLE  3.7 

• 

.70 

SYSTEM  EFFECTIVENESS  VALUES  FOR  VARIOUS 
OPERATOR  PERFORMANCE  EQUATIONS 
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CHAPTER  4 
RESULTS  AND  CONCLUSIONS 

4.1  Introduction 

In  this  chapter,  the  conclusions  of  this  work  will  be  discussed.  Also 
contained  in  this  chapter  is  a  discussion  of  the  sensitivity  of  Lie's 
System  Effectiveness  model  when  the  various  operator  performance  equations 
are  utilized  in  the  simulation.  There  is  also  an  analysis  comparing  the 
mean  operator  performance  for  a  small  number  of  missions  in  the  same  region 
of  the  LFL  curve  with  the  mean  performance  value  for  the  entire  cycle  of  the 
LFL  curve.  The  last  section  of  this  chapter  will  outline  the  requirement 
for  possible  future  investigations  in  the  area  of  training/learning  and 
their  impact  on  operator  performance  and  the  subsequent  effect  of  operator 
performance  on  the  effectiveness  of  a  system. 

4.2  Summary  and  Discussion  of  Results 

In  this  section,  the  findings  of  Chapter  3  are  summarized  and  analyzed. 
The  sections  of  Chapter  3  which  are  of  interest  here  are:  3.3,  3.4.1,  and 
3.4.2. 

4.2.1  Sensitivity  Analysis  of  Lie's  SE  Model 

Referring  back  to  Table  3.7  which  lists  values  of  System  Effectiveness 
for  the  various  expressions  of  operator  performance,  it  can  be  seen  that 
the  values  for  System  Effectiveness  definitely  depend  upon  which  equation 
for  operator  performance  is  utilized  in  the  simulation  program.  In  other 
words,  System  Effectiveness  is  a  function  of  operator  performance  when  the 
expression  for  operator  performance  is  used  in  conjunction  with  the  simu- 
lation. 
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The  resultant  System  Effectiveness  values  for  the  Log  Pseudo- 
Learning  and  Cubic  Pseudo-Learning  curves  (.56  vs  .58)  are  relatively  close 
together  as  would  be  expected  by  comparing  the  shapes  of  the  two  curves  in 
Figures  2.1  and  2.2.  They  both  represent  increasing  performance  functions 
with  the  only  difference  being  that  the  Cubic  curve  reaches  the  asymptote 
of  1.0  faster  than  the  Log  curve.  This  explains  why  the  System  Effectiveness 
value  of  the  Cubic  curve  (.58)  is  slightly  larger  than  that  of  the  Log 
curve  (.56). 

The  System  Effectiveness  index  corresponding  to  the  Learn-Forget-Learn 
(LFL)  curve  (SE  =  .42)  is  significantly  less  than  the  SE  values  of  the  Log 
and  Cubic  curves  (.56  and  .58).  The  reasoning  behind  this  difference  in 
values  can  again  be  explained  by  comparing  the  shapes  of  the  three  curves 
(Figures  2.1,  2.2,  and  2.4).  The  LFL  curve  portrays  an  increasing-decreasing- 
increasing  function  of  performance  while  the  other  two  curves  are  strictly 
increasing  functions  of  performance.  Because  the  LFL  curve  has  a  decreasing 
performance  section,  this  explains  the  smaller  value  of  System  Effectiveness 
for  this  particular  curve. 

Lie's  expression  for  operator  performance,  y  =  y->  +  y2e~6t,  that  he 
utilized  in  the  simulation  yielded  a  System  Effectiveness  value  of  .52. 
Even  though  this  expression  is  a  decreasing  function  of  performance  between 
retraining  periods,  it  still  produces  a  relatively  high  index  of  System 
Effectiveness.  This  is  because  the  curve  starts  at  a  performance  value 
close  to  100  per  cent  and  decreases  from  there  to  a  operator  performance 
value  of  92  per  cent  which  is  large  compared  to  the  performance  values 
of  the  Log  and  Cubic  curves. 
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When  the  constant  values  of  .50  and  1.0  for  operator  performance  are 
utilized  in  the  simulation,  they  produced  a  range  of  System  Effectiveness 
values  from  .32  to  .70.  It  should  be  noted  that  the  other  four  values  for 
System  Effectiveness  fall  between  the  values  of  .32  and  .70. 

From  the  above  discussion,  it  is  clear  that  System  Effectiveness  is 
very  sensitive  to  the  various  equations  that  express  operator  performance. 

4.2.2  Comparison  of  Mean  Performance  Values  Obtained  from  LFL  Curve 

The  mean  operator  performance  values  calculated  in  sections  3.4.1 
and  3.4.2  are  summarized  in  Table  4.1.  Note  that  the  grand  mean  calculated 
from  the  means  of  the  four  samples  is  approximately  the  same  as  the  mean 
calculated  by  the  integration  method.  As  was  mentioned  in  section  3.4.2, 
this  result  is  not  surprising  because  the  sample  population  of  values  were 
utilized  to  calculate  the  two  means.  The  same  logic  applies  to  the  fact 
that  the  two  System  Effectiveness  values  are  approximately  the  same. 

The  most  important  result  obtained  from  the  method  of  mission  interval 
samples  is  that  the  means  of  the  individual  samples  are,  in  most  cases, 
significantly  different  than  the  overall  mean  value;  that  is,  .55,  .54, 
.57,  and  .70  are  significantly  different  than  .60.  This  implies  that  the 
overall  mean  can  be  used  to  estimate  the  average  System  Effectiveness 
if  a  large  number  of  missions  are  to  be  considered,  but  if  the  average 
System  Effectiveness  for  a  small  interval  of  missions  is  required,  the 
overall  mean  performance  value  can  not  be  utilized.  When  a  small  interval 
of  missions  is  to  be  studied,  the  average  operator  performance  has  to  be 
obtained  by  consulting  the  portion  of  the  LFL  curve  which  applies  to  the 
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mission  interval  under  study.  The  average  operator  performance  value 
obtained  from  the  applicable  portion  of  the  LFL  curve  then  can  be  utilized 
to  calculate  the  average  System  Effectiveness  for  the  specific  interval  of 
missions  under  consideration. 

4.2.3  Summary  of  Results 

Table  3.7  demonstrated  that  the  System  Effectiveness  model  developed 
by  Lie  (16)  is  sensitive  to  changing  operator  performance  expressions 
which  are  utilized  in  the  simulation. 

It  also  was  shown  that  the  overall  mean  performance  value  of  the  LFL 
curve  can  be  utilized  to  estimate  an  average  System  Effectiveness  value 
when  a  large  number  of  missions  are  to  be  considered.  But  it  was  also 
demonstrated  that  the  LFL  curve's  mean  performance  value  could  not  be  used 
to  obtain  an  average  System  Effectiveness  value  if  only  a  small  sample 
of  missions  was  to  be  studied.  In  this  type  of  situation,  the  average 
performance  had  to  be  obtained  directly  from  the  LFL  curve. 

4.3  Proposed  Future  Investigations 

Because  of  the  absence  of  any  significant  research  in  the  area  of 
operator  training/learning  and  their  subsequent  effect  on  operator  per- 
formance, the  field  is  open  to  any  number  of  studies  that  can  be  developed 
in  this  area. 

First  and  foremost,  a  consistent  and  reliable  source  for  operator  per- 
formance data  should  be  identified.  Without  operator  performance  data,  the 
plotting  of  training/learning  curves  would  be  impossible,  and  if  the  curves 
can  not  be  obtained,  there  can  be  no  analytic  expression  for  operator  per- 
formance developed.  Gephart  and  Balachandran  (10)  suggested  that  human 
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performance  data  could  be  obtained  from  the  training  programs  or  proficiency 
evaluations  of  the  operators  whose  performance  was  of  interest.  This 
suggestion  appears  to  be  logical  and  should  warrant  further  research  in  the 
areas  of  training  programs  and  proficiency  evaluations  to  ascertain  if 
they  would  constitute  a  good  source  for  operator  performance  data.  Surely, 
there  are  other  sources  of  operator  performance  data  which  can  be  identified 
and  utilized,  and,  if  at  all  possible,  the  data  should  be  expressable  in 
terms  of  operator  performance  versus  time.  The  utilization  of  these  specific 
units  (performance  vs.  time)  would  facilitate  the  inclusion  of  the  operator 
performance  expression  into  all  the  System  Effectiveness  simulation  models. 

Lie's  simulation  model  is  rather  generalized  in  its  formulation.  The 
development  of  models  which  are  more  specific  in  their  formulation  and 
which  can  be  applied  in  detail  to  a  particular  system  is  also  proposed  as 
a  possible  future  investigation. 

Also,  in  Lie's  simulation  it  was  assumed  that  the  probability  of  mission 
success  due  to  environmental  conditions  was  a  constant  in  each  environmental 
condition.  Furthermore,  the  probability  of  mission  success  due  to  the 
operator  was  assumed  to  be  independent  of  the  environmental  conditions. 
In  actuality,  the  performance  of  an  operator  is  almost  certain  to  be 
affected  by  the  environmental  conditions  in  which  the  operator  has  to  per- 
form. In  other  words,  an  operator  is  likely  to  perform  at  a  higher  level 
in  good  weather  conditions  (moderate  temperatures,  low  humidity,  etc.)  than 
in  bad  weather  conditions  (high  or  low  temperatures,  mud,  snow,  etc.). 
The  dependence  of  operator  performance  on  the  environmental  conditions,  or 
the  conditional  probability  of  operator  performance  given  a  certain  environ- 
ment, is  an  area  that  needs  to  be  researched. 
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Once  a  source  of  operator  performance  is  identified,  then  analytical 
expressions  for  operator  performance  can  be  formulated.  Research  should  be 
conducted  in  the  area  of  applying  these  operator  performance  expressions  to 
the  various  other  Systems  Effectiveness  models  which  were  described  in 
references  (10)  and  (14).  The  sensitivity  of  these  models  to  various  ex- 
pressions for  operator  performance  should  also  be  studied. 
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Method  Utilized     Average  Performance  (%)  System  Effectiveness 
Integration  62  .38 

Mission  Interval  Samples  60  .40 

10th  Interval  59 

20th  Interval  54 

30th  Interval  55 

40th  Interval  70 


TABLE  4.1 
SUMMARY  OF  OPERATOR  PERFORMANCE  VALUES 


71 


REFERENCES 

1.  Alden,  R.  J.,  "Learning  Curves:  An  Example",  Journal  of  Industrial 
Engineering,  December,  1974. 

2.  Askren,  W.  B.,  "Symposium  on  Reliability  of  Human  Performance  in  Work", 
1966  Annual  Convention  of  the  American  Psychological  Association, 
September,  1966. 

3.  Askren,  W.  B.  and  Regulinski,  T.  L.,  "Quantifying  Human  Performance  for 
Reliability  Analysis  of  Systems",  Human  Factors,  Vol.  11,  No.  4,  1969. 

4.  Askren,  N.  B. ,  "Quantifying  Human  Performance  Reliability",  AFHRL-TR- 
71-22,  Air  Force  Human  Resources  Laboratory,  Wright- Patterson  Air  Force 
Base,  Ohio,  June,  1971. 

5.  Buck,  J.  R.,  Tanchoco,  J.  M.  A.,  and  Sweet,  A.  L.,  "Parameter  Estimation 
Methods  of  Discrete  Exponential  Learning  Curves",  AIIE  Transactions, 
Vol.  8,  No.  2,  1976. 

6.  Carlson,  J.  G.,  and  Rowe,  A.  J.,  "How  Much  Does  Forgetting  Cost?", 
Journal  of  Industrial  Engineering,  September,  1976. 

7.  Cochran,  E.  B.,  "New  Concepts  of  the  Learning  Curve",  Journal  of  Industrial 
Engineering,  July-August,  1960. 

8.  Corlett,  E.  N. ,  and  Morcombe,  U.  J.,  "Straightening  Out  Learning  Curves", 
Personnel  Management,  Vol.  2,  No.  6,  June,  1970. 


72 


9.  Daniels,  R.  W.  "Factors  Affecting  Industrial  Learning  on  Interrupted 
Production  Schedules",  Master's  Thesis,  Kansas  State  University,  1966. 

10.  Gephart,  L.  S.,  and  Balachandran,  V.,  "A  Stochastic  System  Effectiveness 
Simulation  Model",  Technical  Report  UDRI-TR-70-12,  University  of  Dayton 
Research  Institute,  June,  1969. 

11.  Glover,  J.,  "Manufacturing  Progress  Functions  I.  An  Alternative  Model 
and  Its  Comparison  with  Existing  Functions",  International  Journal 

of  Production  Research,  Vol.  4,  No.  4,  1966. 

12.  Goel ,  S.  N.,  and  Becknell,  R.  H.,  "Learning  Curves  That  Work",  Journal 
of  Industrial  Engineering,  May,  1972. 

13.  Hancock,  W.  M. ,  "Progress  Report  on  Learning  Curve  Research",  MTM 
Journal,  Vol.  9,  No.  2,  1963. 

14.  Hanifan,  D.  T.,  "Navy  System  Effectiveness  Manual,  NAVMAT  P3941-B 
Proposed",  The  Harbinger  Corporation,  Santa  Monica,  California, 
October,  1973. 

15.  Lerch,  J.  F.,  and  Buck,  J.  R. ,  "An  Exponential  Learning  Curve 
Experiment",  Proceedings,  19   Annual  Meeting  of  the  Human  Factors 
Society,  Dallas,  Texas,  1975. 

16.  Lie,  C.  H.,  "System  Effectiveness  Models  for  Maintained  Systems: 
Analytic  and  Simulation  Approach",  Doctor's  Dissertation,  Kansas 
State  University,  1977. 


73 


17.  Meister,  D.,  "Methods  of  Predicting  Human  Reliability  in  Man-Machine 
Systems",  Human  Factors,  Vol.  6,  No.  4,  August,  1964. 

18.  Meister,  D. ,  Comparative  Analysis  of  Human  Reliability  Models, 
Bunker-Ramo  Electronics  Systems  Division,  Westlake  Village,  California, 
1971. 

19.  Meister,  D.,  "Subjective  Data  in  Human  Reliability  Estimates",  1978 
Reliability  and  Maintainability  Symposium,  Los  Angeles,  California, 
January,  1978. 

20.  Nanda,  R.,  and  Adler,  6.  L.,  Learning  Curves  Theory  and  Application, 
American  Institute  of  Industrial  Engineers,  Inc.,  Norcross,  Georgia, 
1977. 

21.  Peters,  G.  A.,  and  Hussman,  T.  A.,  "Human  Factors  in  Systems  Reliability", 
Human  Factors,  Vol.  1,  No.  2,  April,  1959. 

22.  Pickrel,  E.  W.,  and  McDonald,  T.  A.,  "Quantification  of  Human  Performance 
in  Large,  Complex  Systems",  Human  Factors,  Vol.  6,  No.  4,  August,  1964. 

23.  Pondy,  L.  R. ,  Available  Methods  for  Evaluating  Man-Machine  Systems, 
Battelle  Memorial  Institute,  Columbus,  Ohio,  September,  1959. 

24.  Rook,  L.  W.,  Jr.,  "Reduction  of  Human  Error  in  Industrial  Production", 
Sandia  Corporation  Technical  Memorandum  SCTM  93-62(14),  Albuquerque, 
New  Mexico,  June,  1962. 

25.  Sandler,  G.  H.,  System  Reliability  Engineering,  Prentice-Hall,  New 
York,  1963. 


74 


26.  Shapero,  A.,  Cooper,  J.  I.,  Rappaport,  J.,  Schaeffer,  R.  H.,  and  Bates, 
C,  Jr.,  "Human  Engineering  Testing  and  Malfunction  Data  Collection  in 
Weapon  System  Test  Programs",  WADC-TR-60-36,  Wright-Patterson  Air 
Force  Base,  Ohio,  February,  1960. 

27.  Swain,  A.  D.,  "Some  Problems  in  the  Measurement  of  Human  Performance 
in  Man-Machine  Systems",  Human  Factors,  Vol.  6,  No.  4,  August,  1964. 

28.  Swain,  A.  D.,  "Technique  for  Human  Error  Rate  Prediction",  Sandia 
Laboratories,  SCR-64-1338,  Albuquerque,  New  Mexico,  August,  1964. 

29.  Swain,  A.  D.,  "THERP",  Proceedings,  Symposium  on  Quantification  of 
Human  Performance,  Albuquerque,  New  Mexico,  August,  1964. 

30.  Topmiller,  D.  A.,  "Mathematical  Models  of  Human  Performance  in  Man- 
Machine  Systems",  AMRL-TR-68-22,  Aerospace  Medical  Research  Laboratory, 
Wright-Patterson  Air  Force  Base,  Ohio,  1968. 

31.  Towill,  D.  R.,  "A  Directed  Method  for  the  Determination  of  Learning 
Curve  Parameters  From  Historical  Data",  International  Journal  of 
Production  Research,  Vol.  11,  No.  1,  1973. 

32.  Williams,  H.  L.,  "Reliability  Evaluation  of  the  Human  Component  in 
Man-Machine  Systems",  Electrical  Manufacturing,  Vol.  6,  No.  1,  1958. 

33.  Wright,  T.  P.,  "Factors  Affecting  the  Cost  of  Airplanes",  Journal  of 
Aeronautical  Science,  Vol.  3,  No.  2,  February,  1936. 


APPENDIX  A 


A-l 


C^**  COMPUTER  SIMULAUON  MODEL  FOR  Tf-S  EVALUATION  OF  MISSION  EFFECTIVENESS  **** 

C...  *****  REQUIRED  INFORMATION  ***** 

C...  REMARK  1 

c"  USERS    ARE    REQUIRED    TC    SUPPLY    THE    FOLLOWING    DATA 

C  MC=NUM6ER    OF    UNITfLlMlTeD    UP    TC    10) 

C  Ka*NUMS£R    OF    M  ISSION tL IMi TED   UP    IC    200) 

w  a  •  • 

C...  REMARK  2 

C**  EIGHT  PROBABILITY  DISTRIBUTIONS  FCK  THE  FOLLOWING  RANDOM  VARIABLES 

C  (K«l,2,3,* ,5,6, 11,12)  ARE  REQUIRED. 

C  K=l  :  TIME  INTERVAL  BETWEEN  MISSION  STARTS 

C  K=2  •    MISSION  DURATION 

C  K=3  :  TIME  INTERVAL  BETWEEN  FAILURE  INDUCED  BY  HARDWARE 

C  K=4  :  DL'.«NTIMSIHA«U«ARE  INDUCED) 

C  K=5  :  TIME  INTERVAL  BETV.EEN  FAILURE  INDUCED  BY  OPERATOR 

C  K=6  :  DCWNT IME1GPERATUR  INDUCED) 

C  K=ll:  ENVIRONMENTAL  CONDITION  ANC  SYSTEM  PERFORMANCE  UNDER  GIVEN 

C  CONDITION 

C  K=U:    MISSION    TYPE 

C  EACH    DISTRIBUTION    IS    REUUIRED    TC    BE    DEFINED    IN    THE    FOLLOWING    FORMAT. 

C  

C  XK(I)               CFKIII 

C  XKil)               CFKU) 

C  XK12J              CFKI2) 

C 

C 

C 

C  XKIN1K?          CFK(NIK) 

C  rfHERE,    XK(I)     IS    THE    NUMERICAL    VALUE    THAT    RANDOM    VARIABLES    CAN     TAKE, 

C  CFK.ll)     IS    The    CUMULATIVE    FREC'JE.NCY  ,    AND 

C  MK    IS    LIMITED    UP    TC    10. 

c  •  *  • 

C...  REMARK    3 

C*"  FOR    EACH    UNIT!  1=1,2,    ...     ,MC)i     THE    FOLLOWING    OPERATOR    PERFORMANCE 

C  INDICES    ARE    kECU [REO- 

C  YH  t)«STEAOY-SfATE    PERFORMANCE 

C  Y2(l)»(  INITIAL    PERFuRMANC£)-Yl(I  i 

C  51  I! "DECREASING    RATE    OF    PERFORMANCE 

C  TP( I)=RETHA1NIMG    PERIOD 

C... 

C...  *****  [NPUI  0AT4  FORMA  I  ***** 

C... 

C  FOR  THE  INPUT  DATA  FGRMAT  AND  CROER, 

C  REFER  TO  THE  -^EAu  STATEMcNTS  ANU  THE  CORRESPONDING  FCRMAT 

C  STATEMENTS  IN  THE  FIRST  h>ART  OF  THE  MAIN  PRCORAM. 

C... 

C...  MAIN    PROGRAM 

DIMENSION    TB.4S  ( 10, 200)  ,CTMS(  1 0,200 >  ,0(10,200)  ,CTMF<  10,200), 

1  TBHF( 10,200)  ,CThrS( 10  ,2  JO) , RT H { IC  ,200 > , CT FRF ( 10,200) , 

2CTHP.F1I  101  ,TYPE(  10,200) ,OEU< 10, 200)  ,KHl(l'J)  ,  Jh  (  10  )  ,  JC  ( 1  C) 
01  MEMS  I  C.N    EWMT  1 10, 200), UP  ( 10,200  1,  YU  1  0  ),  Y2(  1C),8(IC),7P(10J, 

1AHI  10,200)  ,RH(1 0,200!  ,RH A ( 10,200)  ,tM{ 10, 200) , EMCI  10)  ,  EMS  I  200) 


A-2 


3 
4 


5 
6 
7 
3 
9 
10 

a 

12 
13 
14 
15 
16 
17 
15 
19 
20 
21 
22 
2i 
24 
25 
26 
27 
28 
29 


30 
31 
32 
33 
34 
35 
36 
37 
38 
39 
40 
41 
42 
43 
44 
45 


46 
47 
43 
49 
50 
51 


700 

710 
750 


DlMENS 
1X6( 10) 

DIMENS 
1CF6U0 

READ    I 

READ I  5 

F C*M AT 
R£AD(5 
FORMAT 
FORMAT 
REAO( 5 
REAC(5 
REAOJ 5 
REAOl 5 
REA0I5 
READ! 5 
RCACl 5 
READ (5 
READ! 5 
REACH  5 
READ( 5 
REAOt  5 
REA01 5 
REA0(5 
REAOf 5 
REAOl 5 
REAOl 5 
REAC15 
READ!  5 
REAO! 5 


C. 

c, 

c. 


CN  XI  (10)  ,X2(10) ,X3(10),X4l 10)  ,X5( 10), 
Xll( 10) ,X12< 10) 

O.N  CF11  10)  ,CF2(  10)  ,CF3<  1G)  ,CF4l  10)  ,CF5UC) 
,CF1H  10)  ,CF12<10) 

PUT    DATA 


730) 
214) 
710) 
314) 
I  OF  8 
750) 
7  50) 
7;0) 
750) 
750) 
730) 
75d) 
750) 
750) 
75(J) 
750) 
7  50) 
750) 
750) 
7  50) 
750) 
75C) 
750) 
750) 
750) 


HC  » MM 

M1,NI2,N13,NI4,N!5,M6,NI11,.NI12 


-3) 

(XI (  I  ) 
1CFK  I 
(X2(! ) 
(CF2(  I 
U3<I) 
ICF31  I 
(X4( I ) 
(CF4( I 
(X5( ! ) 
(CF5(  I 
(X6( I ) 
(CF6( I 
( X  1 1  (  I 
(CF11  ( 
(X12( 1 
(CF12  ( 
(YUI  ) 
(Y2( I ) 
(3(  H  , 
<TP( I ) 


1=1, Nil) 
I=l,NU) 
=  1  ,NI2) 

1=1, m 2) 

=  1,.NI3> 
1=1 ,NI3) 
=  1,M4) 
I  =  1  ,  N 1  4  ) 
=1,N15) 
1=1, N15) 
- 1  ,  N 1 6  ) 
1=1, N16) 
1=1, NI  11) 
,1  =  1  iNlil) 
i  =  l,NU2) 
,  1*1, NU2) 
■  I, MO 
=  l,MC) 
=1 ,MC) 
,1=1, MC) 


GENERATE  TIME  INTERVAL  6ETW6EN  CI  S3  ION  STARTS 

1X1=11 

00    10     1=1, MC 
1X1=1X1+10 
00    10    J=l,KW 
CALL    RAfiCU(IXl,IYl,YFLi) 
CALL    0IST.MX1,CF1,YFL1,RNI,NI1) 
T3MSI  I  ,J)=RN1 
i  X  1=  t  Y  I 
10   CONTINUE 

WRITE (6,500) 

500  FORMAT  I '  1' , 'TABLE    A-    TIME     INTERVAL    BETWEEN    MISSION    STARTS'/) 
WRITE (6,  502) (  I  ,1=1  ,MC) 

502FORMATI'     '  ,  15X  ,  '  UNI  T  •  /  '     •  ,  '  Ml  SS  IC.N  '  ,  131  12) 

00    50  5    J=l,.MM 
505   WRITEUtSOl)    J  ,  i  T3HS  (  I  ,  J)  ,  1  =  1  ,MC) 

501  FORMAT!'     '  ,  17  ,  10F 12.2 ) 

..      COMPUTE    ACTUAL    MISSION    STARTING    T  LIES  ( C  LOCK    UPES) 

>    • 

DO    20    I=l,MC 
CTMSIN=0. 
OC    20    J=l,MH 

CTMSI  I ,J)=CTMSIN  +  TBMS( I  ,J) 
CTMSIN=CTMS(I  ,J) 
20   CONTINUE 


A-3 


52 
52 
5h 
55 
5^ 


57 
53 
59 
60 

ti 
62 
o3 
64 
65 
6o 
67 

6a 

69 
70 


7L 
72 
73 
74 
75 
7o 

77 
7? 
79 


20 
3L 

32 
53 
64 
3  5 

CO 

67 
38 
39 
90 

91 
92 

93 


WRITS (6, 510) 
510   FORMAT! '1' , 'TABLE    B.    ACTUAL    MISSICN   STARTING    T  I.MES ( CLDCK    TIMES)'/) 

WRITE16, 50211  I  ,1  =  1, .MC) 

00    515    J=l,«M 
515    WRITE16.501)    J , (CTMSC I , J) i I»l »«Cl 

C...      GENERATE    IDEAL    .MISSION    DURATIONS 

C  •  •   * 

1X2=21 

oc  30   r-t ,mc 

1X2=1X2+20 

DC    30    J«1,MM 

CALL    ?.ANDU<  IX2.IY2.YFL2) 

CALL    DISTNIX2,CF2.YFL2»KN2,NI2) 

D<  I, J)=RN2 

i  X2=IY2 
30    CONTINUE 

WRITEI6.520) 
520    FORMAT!  •  1'  ,  'TAdLE    C.     IDEAL    MISSION    CURAT  I CNS» / ) 

WRITE(6,5C2)<  I,I  =  l,MC) 

OC    52  5    J=l  ,."M 
525   WRITE!6,5011    J.  {  0(  I  ,J»  »I*  WMC) 

C...       COMPUTE    IDEAL       MISSION    FINISHING    TIMSSICIQCX    TIMES) 

<*  •  •    • 

DO    40    1=1, MC 

DC    40    J=l,Mf 

CTMF(  I ,J)=CTMSl I ,J)*0(I,J) 
40   CONTINUE 

WRlTE!6t530) 
530    FORMAT! «1'» 'TABLE    D.       IDEAL   MISSICN    FINISHING    I.IMESICLCCX    TIMES)'/ 
1) 

WRITEU.502H  i  ,!=l  ,MC) 

OC    53  5    J* I, MM 
535    WRITE!6i50l)    J  ,  ( CTPF i  I  ,j )  ,  1  =  1 ,MC) 

C...       GENERATE    MISSICN    TYPES 

L  •  •  • 

1X12=121 

OC    1 1  10    1=1, MC 

I X12=IX12*1C0 

DO    1110    J=l,MM 

CALL    RANCUt 1X12,  IY12.YFL12) 

CALL    0ISTN(X12,CF12,YFL12,K.N12,M12) 

TY?E(  I,J)=RM2 

I X12=I Y12 
1110    CONTINUE 

WRITE(6,1300) 
1300    FORMAT! ' 1' ,' TABLE    E.    MISSICN    TYPES'/) 

WRITE(6,502H  I  ,I  =  1,MC) 

DO    1310    J=l,MM 
1310    WRITE(6t501)    J , ( TYPE  I  I  ,  J )  ,  I =1 ,MC) 
C. 
C. 
C. 
C. 

c. 

cc. 
c 


94 


GENERATE  TINE  INTERVAL  BETWEEN  FAILURES  AND  DURATION  CF  REPAIR 
COMPUTE  ACTUAL  FAILURE  STARTING  TIMES  A.ND  ACTUAL  REPAIR  FINISHING 
TIKESICLCCK  TIMES) 

.  FUR  THE  FAILURE  INCUCEO  -3Y  HARDWARES 

WRITE(6,53S) 
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95  53<*    FGRMATl 'l' , 'THE    FOLLOWING    TABLES    SHCW    •/'     '  ,'TIPE    INTERVAL    BETWEEN 

1  FAIL'JRESlTfHF)  ,  '/'  ',  'ACTUAL  FAILURE  STARTING  T  I  H£S  <CTHFS  )  ,  '  /  '  ', 
2'DURAT1CN  CF  REP  MR (RTH) , • / '  ' ,  'AND  ACTUAL  kEPAIR  FINISHING  TIfESi 
3CTHRF)'/'     ','FOR    THE    PAiLURE    INCUCEC    t>Y    hARCVARfcS') 

96  1X3=31 

97  1X4=41 

93  DO    540    1*1, MC 

99  fcR ITS (fc, 541)1,1 

100  541    FORMAT! '1'  , 'TABLE    F. ', 12, • .HARDWARE    INCJCEO ' , I  3// '     ',  'FAILURE    AMJ • 

!/'     ',  'REPAIR    INDEX'  , ex.'TBHF'  ,7X,»CTHFS'  , 9X,'RTh« ,7X  ,'CTHRF'l 

101  CTHFSZ'O. 

102  KTHZ-O. 

103  1X3=1X3+100 

104  1X4=1X4+100 

105  J=l 

106  545    CALL    R5FA IL I  I X3,  1X4 , I Y3, 1 Y4 , X3 ,CF3 ,M 3, X4 ,CF4 ,N  14  , 

lTShFJ,RTrtD,CTHFSC,CTHP.rJ,CTHr$Z,RTHZ) 

107  TBHFl  I  ,J)  =  ToHFO 

108  CTHFSl l,J)=CTHFSO 

109  ?.TH(  I  ,JJ=RTHO 

110  CTHRFt  I,J)=CTrtRFD 

111  JH(1)=J 

112  WRITE (6, 542) J , TBHFl  I, J  I  , CTHFSl I ,J),RTK(  I, J  J  ,CTHRF (  I, J) 

113  CThFSZ=CTHFS( I  ,J) 

114  RThZ=RTH(I,J) 

115  IFICTHFSZ.GE.CTMF! I  ,MMJ J    GO    TO    540 

116  J  =  J  +  l 

117  IX3=IY3 

118  IX4=IY4 

119  GC    Tu    545 

120  540   CONTINUE 

121  5^2    FORMAT!'     ' , I 12 , 4F 12 .2) 

CC...  FCR  THE  FAILURE  INCUCFD  BY  CPERATORS 
C... 

122  WRITE(6,550) 

123  550  FORMAT  I ' 1'  , 'THE  FOLLOWING  TABLES  SHOW'/'  ' ,'TIfE  INTERVAL  BETWEEN 

1FAILURESJTBCF) , •/■  ',  'ACTUAL  FAILURE  STARTING  T IMESICTOFS) , •/ •  ',' 
2DURATICN  CF  REPAIR  (RTC  J ,«/ '  '  ,'A.gC  ACTUAL  REPAIR  FINISHING  TlaEStC 
3T0RFJ  »/•  '.'FUR  THE  FAILURE  INDUCED  BY  OPERATORS') 

124  1X5=51 

125  1X6=61 

126  00    5al    1=1, MC 

127  WRITEI6, 552)1  ,1 

128  552    FORMAT!  •  I'  , 'TAnLE    G. ' , 1 1 , ' .OPER ATCR    INDUCED1 , 13// '     '  , 'FAILURE    AND' 

1/ ■     ',  'REPAIR    INDEX', 8X, ' T30F' ,  7X,  'CTOFS'  ,  SX.'RfC  ,7X,'C7CRF'} 

129  CTCFSZ=0. 

130  RTGZ=0. 

131  iX5=IX5+100 

132  1X6=1X6+100 

133  J=1+JHII) 

134  555    CALL    3EF A  I L t I X5 , 1X6 ,  IY 5 , I Y6,X5 , CF 5, N I  5, X6, CF6 ,N  16 , 

ITBOFD,RTOO»CTOFSD,CTORFO,CTOFSZ,RT02) 

135  TGhF!  I,J)  =  T80r-0 

136  CTHFS! I ,J)=CTCFSO 

137  RTM  I  ,JJ=RTGD 

138  CTHRF{ I,J)«CTCRFO 

139  JUU)=J 

140  rfRlTE(6,542)J , TBHFl  I, J) , CTHFSl I  ,J ) ,KTH(  I  , J)  ,CTHRF!I  ,  J) 

141  CTCFSZ  =  CTHFS(  I  ,J) 
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142  RTOZ=RTH< I, J) 

143  IF (CTCFSZ.GE.CTMFC  I, MM) J    Go    TO    551 

144  J=J+I 

145  IX5=IY5 

146  IX6=IY6 

147  GO  TO  555 

148  5  51  CONTINUE 

C...   REARRANGE  FAILURE  STARTING  TIMES  AND 
C...   REPAIR  FINISHING  TIMES  IN  ASCENDING  ORDER 
C... 

149  00  1000  1=1, MC 

150  KA=J2(I>-1 

151  OC  1000  II=1,KA 

152  JA=JO(I)-i 

153  00  1000  K»l,JA 

154  IFICThrSt  I  ,K+1).GE.CTHFS(  I ,K)  )  GO  TC  1001 

155  TE.»!?=CThFS(  I,K) 

156  CTHFSI  I,K)-CTHFS(  I  ,XU) 

157  CThFSU,K*l)«TEMP 

153  1001    IFICTHRF1 I ,X+l J.GE.CTHRFI I,K)  )    GO    TO    1000 

159  T5MP1=CTHRF(I ,KJ 

160  CTHkF(I,X)»CThRF( I,K+1J 

161  CTHRFti ,K*l)=TEMPl 

162  1000  CONTINUE 

C...   PERFORMANCE  OF  UNIT  IN  DIFFERENT  ENVIRONMENT 
C... 

163  90    1X11=111 

164  DC    100    I-l.KC 

165  1X11=1X11+100 

166  DC    LOO    0=l,.*M 

167  CALL    RA.NCUUXU,  IYU.YFLil  ) 

168  CALL    QISTNUU,CH1,YFL11,RNI1,NI  11) 

169  ENVMTC!,J)»RNll 

170  IXllalYll 

171  100    CONTINUE 

172  WRITE16.530) 

173  530   FORMAT! » I* , 'TABLE    H.    PERFORMANCE    OF    UNIT    FOR    GIVEN    EAV1 RONMSNT V i 

174  WRITSI6, 502)1  I ,1  =  1, MC) 

175  DO    586    J-l.MM 

176  586  WR1TS16,  SOD    J  ,  <  ENVMTI  I ,  J  )  ,I»l ,  MC  ) 

C...      OPERATOR    PERFORMANCE    CURING   MISSION 

L  «  «  • 

177  DC   300    1  =  1  ,fC 

178  TPI=0. 

179  TPF=TP(l) 

ISO  00    300    J=l ,MM 

131  If (CTMH I ,J).LE.TPF)    GO    TO    310 

1S2  CS?2=CT"1S(  I,J)-T?I 

183  CFP2  =  CTWF( I  ,J)-TPF 

164  TPT=TPF-TPI 

185  OP (I , J)=Y1 ( II+(Yl(  I)/TPT*Y2( I ) / 51  I )*!  l.+£XP(-B(  I)*CSP2>- 
ltXP(-3(! )*CFP2)-cXP(-3II)*TPT) ) )/D( 1,0) 

186  r?[  =  TPF 

187  TPF  =  TPI  +  TP(  I) 

188  GO    TO    300 

189  310    CSP=Cr,VS(I  ,J)-TPI 

190  CFP=CTMF(I,J)-rPl 
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191 
192 

193 
194 
195 
196 
197 
193 


199 
2  CO 
201 
2  02 
2C3 
2  04 
205 
206 


207 
2C8 
2  09 
210 
211 
212 
213 
2  14 
215 
216 
217 
218 


219 
2  20 
221 
222 

2  23 
224 
225 
226 
227 
223 
229 

230 
231 
232 
233 
234 
235 

236 


DPI  I,  J)  =  YK  I)  <-Y2(D/(3(I)*D(  I  ,J))*(EXP[-3U  )*CSP)-EXFl-B(I)*CFF)) 
300  CONTINUE 

WRITE<6,620) 
620  FORMAT  I 'l'» 'TABLE  I.   QPEKATGk  PERFORMANCE  DURING  MISSION'/) 

WRITEI6, 591)1  I  .1  =  1 ,MO 
591  FURMATl'  ',11X, 'OPERATOR' /•  *  .  ' MI SSION '  ,  101 12 ) 

00  625  J*l ,MM 
625  WRITE(fci50I)  J, (CPU ,J>  ,1=1, MO 
u  •  •  • 
C...   COMPUTE  EXPECTED  MISSION  CURAT ICNS 

00  1100  1=1, MC 

00  1100  J=l ,MM 
1100  DEOl I  ,J)=0( l,JS/(ENVMT(I ,J)*OP(  I,  J)  ) 

WRITE (6, 15 JO) 
1500   FORKATI' I'.'TACLE    J.    EXPECTED    MISSION    DJRATICNS'/J 

*P.ITE16,5G2H  I  ,  1  =  1, ^d 

DC    15  50    J»ltMM 
1550    WRITE(6,501)     J  ,  ( OEO ( I  ,  J ) , 1  =  1 , MC ) 

C...       COMPUTE    EXPECTED    MISSION    FINISHING    TIMES 
C... 

DO    1200    I«l,MC 
DO   1200   j«i,mm 

IFJDEOl  I  ,J).LE.TYPE( I, J))     GO    TO    1210 
CTMF!  1  ,J)  =  CT,HS(  l,J)+TYPEl  I,  J  J 
GC    TO    1200 
1210    CTHFCIf J)«CTMS1 I  ,  J>*D£Ot I , J) 
120C    CONTINUE 

WRITEffc,l60C) 
1600    FORMAT (• 1« , 'TABLE    K .    EXPECTED    MISSION    FINISHING    TIMES'/) 
WRITEI6,502)( I ,1=1 ,MC) 
DC    1650   J«tf.MM 
1650    WRITE(6,5G1)     J, ( CTMF (  I , J )  ,  1  =  1 ,MC) 

C. 

c. 

cc. 
c. 


COMPUTE  MISSION  RELIABILITY  AND  4VAILA6ILITY 
INITIALIZATION 


DC  MO    l^l,MC 

CThRFU  I)  =  0. 

KiHlI  I)=l 
110    CONTINUE 
C... 

DO    200    J=1,MM 

00   200    X=l,."C 
1-976    IFICTMS(K.J)  .LE.CTHKF (K,KH1 (K)  )  )     GO    TO    210 

CThRFl(K)  =  CThRFlK,KHl(!<)  ) 

K.H1IK)  =  KHHK)  f-1 

GO    TO    1976 
210    Ir( (CTMSIK.J)  .GE-CThRF  ICO  ) .AND  . < CTMS IX , J )  -LT  . 
ICTHFS'.K.KHKK)  )  )  )     GO    TC    220 

AH(K, J)=0. 

RHIK, J)=0. 

P.HA(K|J1»0. 

GO  TO  200 
220  AH(K, J)=l. 

IF! I CTMF (K, J) .CT.CTHRFlf K) ) .AND . ( CTMF 1 K  ,  J ) . LE. 
lCThFSIK,KHl IK) ) ) )    GO    TO    230 
240    RHU,J)  =  0. 
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237 

23b 
239 
2  40 
241 
242 
243 
2  44 
245 
246 
247 
243 
249 
250 
251 
252 
253 


2  54 
255 
2  56 
257 


258 
259 
260 
261 
262 
263 
264 
265 
266 
267 

266 
269 
270 
271 
272 

273 
274 
275 
2  76 
2  77 
27a 


230 

L120 
200 

535 
581 
583 
600 


605 
602 

C... 


RHA(K,J)=CTFFS(K,KH1(K)  >-CTMS(K,J) 

GO    TO    200 

IFICEQIK.J)  .LE.TYPE(K.J))     GO    TO    1120 

RH(K, J)*0. 

RHA(K,  J)=TYPE(K,J) 

GO  TO  200 

RH(K»J)-1. 

RHA(K,J)=0EC(K,J) 

CONTINUE 

WRITE16.535) 

FORMAT! 'I*  ,'TASLE  L.  AVALAGILITY  GF  EACH  UNIT'/) 

WRITE(6,53U(  l,[*l,MCJ 

FORMUl'     ',15X,'UNIT  •/•     '  ,  '  M  <  SS  ION  '  ,  101  12) 

DC    533    J=l ,MM 

*RlTE!ot501)    JtUH(I,JJ,I»liMCJ 

WRITE (6,600) 

FORMAT!  '  I'  , 'TABLE    M.       MISSION    RELIABILITY    OF    WIT  '/•     ',10X,'N0 

1TE:  IMPLICATION  OF  (A)  •/'  ',15X,»A  IS  THE  CURATIC.N  CF  MISSION  PER 
2ICD    CARRIED    OUT    BY    UNIT'/) 

WRITE {6,561)1  I  ,I=1,MC) 

00    60  5    J»l,MM 

WRI  IE  (6, 602)     J,(RH(I  ,J)  ,P.HA<  I  ,J)  ,  1  =  1, NO 

FORMAT!  •     • ,17,  10 (F 3.0, •(  ■  ,F7.2,  ')  ') ) 

COMPUTE    MISSION    EFFECTIVENESS 


410 

400 

630 

635 

650 

651 
5000 


C. 

c. 


DC  40 
SUMC  I 
00  41 
EMI  I, 
SUMC  I 
CCfsTI 
EMC!  I 
CONTI 
-.iRITE 
FORMA 

1MISSI 
WRITE 
00  63 
WRITE 
WPvITE 
FORMA 

1) '//' 
V,'R  !  Tc 
FORMA 
WRITE 
FORMA 
STCP 
END 


0    1  =  1, 

=  0. 

0    J=l, 

J)=Ah( 

■SUMC i 
NUE 

>=SUMC 
NUE 
16,630 
T(  '  I'  , 
ON'/) 
(6,502 
5  J*l, 
(6,50  1 
( 6  ,650 
71  •  1'  , 
'  ,7X, 
(6,651 
Tl  '  '  , 
(6,500 
T  !  '  1  •  ) 


KC 

MM 

I, J)*RH(I  ,J) 

♦EMU  ,J) 

I/MM 


) 

•TABLE    N.       MISSION    EFFECTIVENESS   OF    EACH    UNIT    FOR    EACH 

)(  I  ,1=1, MC) 

MM 

)     J,  IEM!  I, J)  ,  1  =  1, MO 

) (  I  ,1=1 ,MC) 

'TABLE   C.       OVERALL    MISSION    EFFECTI  VSf\ESS    CF    UNIT    (Me(I) 

10112) 

)  (  E  MC  (  I  )  ,  I  =•  1 ,  MC  ) 

7X,10F12.2) 

C) 


GENERATE    RANDOM    NUMBERS 


2  79 

280 
281 
282 
283 
2  84 
2  85 


SUBROUTINE    RANCUt 1X.IY.YFL) 

IY=tX*65539 

IF! !Y) 5,6,6 

5  IY=IY+2147433647+l 

6  YFL=IY 
YFL=YFL*.4656613E-9 
RETURN 
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2S6 


t  *  •  • 

L   •    •    - 
\f  U   •    • 


END 

FIT   RANCCM    NUMBERS    INTC    D  1ST?.  I  3UT  ION 


2S7  SUBROUTINE    CI STN { X. CF ,RV, RN,N  I  ) 

2  83  DIMENSION    X(.NI),CF<M1 

289  RV=1JQ.*RV 

290  IF  (RV.Le.CF(l)  ),  GG    TO    20 

291  1=2 

292  40    J-I-l 

293  IF  URV.GT.CFl J) )  .  ANG  .  (RV .LE.CF <  1) ) )    GO    TO    30 

294  1=1+1 

2  95  GO    TO    40 

296  20   RN=X(1) 

297  GO    TO    100 

298  30    RN=X(I) 

299  100    RETURN 

300  END 


301 

3  02 
303 
304 
30S 
306 
3  07 
3C8 
309 
310 
311 
312 


C. 

C . . 

c. 

.- 

■-  •   * 

c. 

C-. 


GENERATE    TIME     INTERVAL    BETWEEN    FAILURES    ANO    DURATION    CF    REPAIR    FOX 

HARDWARE    ANC   OPERATOR    INJUCEJ    FAILURES. 

COMPUTE    ACTUAL    FAILURE    STARTING    TIMES    AND    ACTUAL    REPAIR    FINISHING 

T  IMtStCLCCK    TIMES) 


SUBROUTINE    P.EF  A  1  L  (  IXS1  ,  IX  S  2  ,  IYS  1,  IYS2,  XS  1 ,  CFS  1  ,NI  S  1  ,  XS2  ,Cf  S2,  N  IS2  , 
IT3SF,RTS.CTSFS»CTSRF,CTSFSI  ,RTSIN) 
OlMtNSIGN    XSUNISl)  ,  CFSISN  IS  I)  ,  XS21NIS2)  ,CFS2(NIS2) 
CALL    RANDUUXSl  ,  IYSl  .YFLS1  ) 
CALL    GISTN(XS1,CFS1,YFLS1,RNS1,M  Si) 
T3SF=RNSl 

CALL    RANDUI IXS2iIYS2,YFLS2) 
CALL    CISTM(.-.S2,CFS2,YFLS2,RNS2,NiS2) 
RTS«RNS2 

CTSFS=CTSFSI+T3SF+RTSIN 
CTSRF  =  CTSFS+P.rS 
RETURN 
ENO 
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ABSTRACT 

In  only  the  last  25  years  has  operator  reliability  been  incorporated 
with  hardware  reliability  to  obtain  a  value  for  the  effectiveness  of  a  man- 
machine  system.  Very  seldom,  and  in  most  cases,  never,  do  any  of  the  human 
reliability  models  address  the  effect  that  operator  learning  has  on  human 
reliability  and  the  subsequent  impact  that  operator  reliability  has  on 
system  effectiveness.  This  research  studied  the  sensitivity  of  a  system 
effectiveness  model  to  changes  in  operator  learning  levels. 

Learning  data  which  was  expressed  in  terms  of  performance  versus  time, 
was  obtained  from  a  paper  which  analyzed  the  performance  of  an  actual  manu- 
facturing task.  This  data  was  utilized  to  develop  three  different  curves  - 
a  log  pseudo-learning  curve,  a  cubic  pseudo-learning  curve,  and  a  Learn- 
Forget-Learn  (LFL)  curve.  Each  curve  expressed  operator  performance  as  a 
function  of  time. 

The  expressions  for  each  of  the  three  curves  were  then  utilized  in 
conjunction  with  a  system  effectiveness  simulation  model  to  formulate 
values  for  system  effectiveness.  The  various  values  of  system  effectiveness 
obtained  from  the  simulation  demonstrated  that  the  model  was  sensitive  to 
changing  levels  of  operator  performance. 

This  research  is  unique  because  this  is  the  first  time  that  operator 
learning  curves  have  been  utilized  in  conjunction  with  a  simulation  of 
system  effectiveness. 


